{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "resident-turkish",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4.01 s (started: 2021-07-25 12:04:35 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 自动计算cell的计算时间\n",
    "%load_ext autotime\n",
    "\n",
    "#设置使用的gpu\n",
    "import tensorflow as tf\n",
    "\n",
    "gpus = tf.config.list_physical_devices(\"GPU\")\n",
    "\n",
    "if gpus:\n",
    "   \n",
    "    gpu0 = gpus[2] #如果有多个GPU，仅使用第0个GPU\n",
    "    tf.config.experimental.set_memory_growth(gpu0, True) #设置GPU显存用量按需使用\n",
    "    # 或者也可以设置GPU显存为固定使用量(例如：4G)\n",
    "    #tf.config.experimental.set_virtual_device_configuration(gpu0,\n",
    "    #    [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=4096)]) \n",
    "    tf.config.set_visible_devices([gpu0],\"GPU\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "immediate-waters",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 10.1 ms (started: 2021-07-25 12:04:40 +08:00)\n"
     ]
    }
   ],
   "source": [
    "%config InlineBackend.figure_format='svg' #矢量图设置，让绘图更清晰"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "korean-essex",
   "metadata": {},
   "source": [
    "# 神经网络入门：分类和回归"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "located-welding",
   "metadata": {},
   "source": [
    "> 本章包含\n",
    "* 真实世界机器学习工作流程的第一个示例\n",
    "* 处理矢量数据的分类问题\n",
    "* 处理向量数据上的连续回归问题"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "charitable-sydney",
   "metadata": {},
   "source": [
    "本章旨在帮助您开始使用神经网络解决实际问题。 您将巩固从第 2 章和第 3 章中获得的知识，并将所学应用于三个新任务，涵盖神经网络的三个最常见用例——二元分类、多类分类和标量回归：\n",
    "* 将电影评论分类为正面或负面（二元分类）\n",
    "* 按主题对新闻线进行分类（多类分类）\n",
    "* 给定房地产数据估计房屋价格（标量回归）"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "regulated-opposition",
   "metadata": {},
   "source": [
    "这些示例将是您第一次接触端到端机器学习工作流：您将了解数据预处理、基本模型架构原则和模型评估。\n",
    "\n",
    "到本章结束时，您将能够使用神经网络处理向量数据上的简单分类和回归任务。 然后，您将准备好在第 5 章中开始构建对机器学习的更有原则的、理论驱动的理解。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "opposed-approval",
   "metadata": {},
   "source": [
    "> 分类和回归词汇表\n",
    "\n",
    "分类和回归涉及许多专业术语。 您已经在前面的示例中遇到过其中的一些，并且您将在以后的章节中看到更多。 它们具有精确的、特定于机器学习的定义，您应该熟悉它们：\n",
    "* 样本或 - 进入模型的一个数据点。 输入\n",
    "* 预测或 - 从您的模型中得出什么结果。 输出\n",
    "* 目标——真相。 根据外部数据源，您的模型理想情况下应该预测的内容。\n",
    "* 预测误差或 — 损失值模型预测与目标之间距离的度量。\n",
    "* Classes - 在分类中可供选择的一组可能的标签\n",
    "* 问题。 例如，在对猫狗图片进行分类时，“狗”和“猫”就是两个类。\n",
    "* 标签 - 分类中类注释的特定实例\n",
    "* 问题。 例如，如果图片 #1234 被注释为包含\n",
    "* 类“狗”，然后“狗”是图片#1234的标签。\n",
    "* Ground-truth 或注释——数据集的所有目标，通常由人类收集。\n",
    "* 二元分类——一个分类任务，其中每个输入样本\n",
    "* 应该归为两个唯一的类别。\n",
    "* 多类分类——一种分类任务，其中每个输入样本应分为两个以上的类别：例如，对手写数字进行分类。\n",
    "* 多标签分类——一种分类任务，其中每个输入样本都可以分配多个标签。例如，给定的图像可能同时包含一只猫和一只狗，并且应该同时使用“猫”标签和“狗”标签进行注释。每个图像的标签数量通常是可变的。\n",
    "* 标量回归——目标是连续标量值的任务。\n",
    "* 预测房价就是一个很好的例子：不同的目标价格形成一个连续的空间。\n",
    "* 向量回归 — 目标是一组连续值的任务：例如，连续向量。如果您对多个值（例如图像中边界框的坐标）进行回归，那么您就是在进行向量回归。\n",
    "* 小批量或 - 由模型同时处理的一小组样本（通常在 8 到 128 之间）。样本数通常是 2 的幂，以方便 GPU 上的内存分配。\n",
    "训练时，小批量用于计算应用于模型权重的单个梯度下降更新。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "specified-trainer",
   "metadata": {},
   "source": [
    "## 电影评论分类：二元分类示例"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "practical-playing",
   "metadata": {},
   "source": [
    "两类分类或二元分类是最常见的机器学习问题之一。 在此示例中，您将学习根据评论的文本内容将电影评论分为正面或负面。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "damaged-ministry",
   "metadata": {},
   "source": [
    "### The IMDB dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "suffering-bonus",
   "metadata": {},
   "source": [
    "您将使用 IMDB 数据集：一组来自互联网电影数据库的 50,000 条高度分化的评论。 它们分为 25,000 条用于训练的评论和 25,000 条用于测试的评论，每组包含 50% 的负面评论和 50% 的正面评论。\n",
    "\n",
    "就像 MNIST 数据集一样，IMDB 数据集与 Keras 打包在一起。 它已经过预处理：评论（单词序列）已经变成了整数序列，其中每个整数代表字典中的特定单词。 这使我们能够专注于模型构建、训练和评估。 在第 11 章中，您将学习如何从头开始处理原始文本输入。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "settled-pavilion",
   "metadata": {},
   "source": [
    "下面的代码将加载数据集（当你第一次运行它时，大约 80 MB 的数据将被下载到你的机器上）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "noble-eligibility",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz\n",
      "17465344/17464789 [==============================] - 22s 1us/step\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<__array_function__ internals>:5: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
      "/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/keras/datasets/imdb.py:155: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
      "  x_train, y_train = np.array(xs[:idx]), np.array(labels[:idx])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 29.7 s (started: 2021-07-22 13:53:47 +08:00)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/keras/datasets/imdb.py:156: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
      "  x_test, y_test = np.array(xs[idx:]), np.array(labels[idx:])\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import imdb\n",
    "\n",
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "composed-spotlight",
   "metadata": {},
   "source": [
    "参数 num_words=10000 意味着您将只保留训练数据中出现频率最高的前 10,000 个单词。 生僻字将被丢弃。 这允许您使用可管理大小的矢量数据。 如果我们不设置这个限制，我们将在训练数据中使用 88,585 个唯一词，这会不必要地大。 许多这些词只出现在一个样本中，因此不能有意义地用于分类。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "short-sailing",
   "metadata": {},
   "source": [
    "变量 train_data test_data 和评论列表； 每个评论都是一个单词索引列表（编码一个单词序列）。 train_labels test_labels 是 0 和 1 的列表，其中 0 代表 1 代表："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "mysterious-deputy",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1,\n",
       " 14,\n",
       " 22,\n",
       " 16,\n",
       " 43,\n",
       " 530,\n",
       " 973,\n",
       " 1622,\n",
       " 1385,\n",
       " 65,\n",
       " 458,\n",
       " 4468,\n",
       " 66,\n",
       " 3941,\n",
       " 4,\n",
       " 173,\n",
       " 36,\n",
       " 256,\n",
       " 5,\n",
       " 25,\n",
       " 100,\n",
       " 43,\n",
       " 838,\n",
       " 112,\n",
       " 50,\n",
       " 670,\n",
       " 2,\n",
       " 9,\n",
       " 35,\n",
       " 480,\n",
       " 284,\n",
       " 5,\n",
       " 150,\n",
       " 4,\n",
       " 172,\n",
       " 112,\n",
       " 167,\n",
       " 2,\n",
       " 336,\n",
       " 385,\n",
       " 39,\n",
       " 4,\n",
       " 172,\n",
       " 4536,\n",
       " 1111,\n",
       " 17,\n",
       " 546,\n",
       " 38,\n",
       " 13,\n",
       " 447,\n",
       " 4,\n",
       " 192,\n",
       " 50,\n",
       " 16,\n",
       " 6,\n",
       " 147,\n",
       " 2025,\n",
       " 19,\n",
       " 14,\n",
       " 22,\n",
       " 4,\n",
       " 1920,\n",
       " 4613,\n",
       " 469,\n",
       " 4,\n",
       " 22,\n",
       " 71,\n",
       " 87,\n",
       " 12,\n",
       " 16,\n",
       " 43,\n",
       " 530,\n",
       " 38,\n",
       " 76,\n",
       " 15,\n",
       " 13,\n",
       " 1247,\n",
       " 4,\n",
       " 22,\n",
       " 17,\n",
       " 515,\n",
       " 17,\n",
       " 12,\n",
       " 16,\n",
       " 626,\n",
       " 18,\n",
       " 2,\n",
       " 5,\n",
       " 62,\n",
       " 386,\n",
       " 12,\n",
       " 8,\n",
       " 316,\n",
       " 8,\n",
       " 106,\n",
       " 5,\n",
       " 4,\n",
       " 2223,\n",
       " 5244,\n",
       " 16,\n",
       " 480,\n",
       " 66,\n",
       " 3785,\n",
       " 33,\n",
       " 4,\n",
       " 130,\n",
       " 12,\n",
       " 16,\n",
       " 38,\n",
       " 619,\n",
       " 5,\n",
       " 25,\n",
       " 124,\n",
       " 51,\n",
       " 36,\n",
       " 135,\n",
       " 48,\n",
       " 25,\n",
       " 1415,\n",
       " 33,\n",
       " 6,\n",
       " 22,\n",
       " 12,\n",
       " 215,\n",
       " 28,\n",
       " 77,\n",
       " 52,\n",
       " 5,\n",
       " 14,\n",
       " 407,\n",
       " 16,\n",
       " 82,\n",
       " 2,\n",
       " 8,\n",
       " 4,\n",
       " 107,\n",
       " 117,\n",
       " 5952,\n",
       " 15,\n",
       " 256,\n",
       " 4,\n",
       " 2,\n",
       " 7,\n",
       " 3766,\n",
       " 5,\n",
       " 723,\n",
       " 36,\n",
       " 71,\n",
       " 43,\n",
       " 530,\n",
       " 476,\n",
       " 26,\n",
       " 400,\n",
       " 317,\n",
       " 46,\n",
       " 7,\n",
       " 4,\n",
       " 2,\n",
       " 1029,\n",
       " 13,\n",
       " 104,\n",
       " 88,\n",
       " 4,\n",
       " 381,\n",
       " 15,\n",
       " 297,\n",
       " 98,\n",
       " 32,\n",
       " 2071,\n",
       " 56,\n",
       " 26,\n",
       " 141,\n",
       " 6,\n",
       " 194,\n",
       " 7486,\n",
       " 18,\n",
       " 4,\n",
       " 226,\n",
       " 22,\n",
       " 21,\n",
       " 134,\n",
       " 476,\n",
       " 26,\n",
       " 480,\n",
       " 5,\n",
       " 144,\n",
       " 30,\n",
       " 5535,\n",
       " 18,\n",
       " 51,\n",
       " 36,\n",
       " 28,\n",
       " 224,\n",
       " 92,\n",
       " 25,\n",
       " 104,\n",
       " 4,\n",
       " 226,\n",
       " 65,\n",
       " 16,\n",
       " 38,\n",
       " 1334,\n",
       " 88,\n",
       " 12,\n",
       " 16,\n",
       " 283,\n",
       " 5,\n",
       " 16,\n",
       " 4472,\n",
       " 113,\n",
       " 103,\n",
       " 32,\n",
       " 15,\n",
       " 16,\n",
       " 5345,\n",
       " 19,\n",
       " 178,\n",
       " 32]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 13.3 ms (started: 2021-07-22 15:16:12 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_data[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "international-plaintiff",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.74 ms (started: 2021-07-22 15:16:32 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_labels[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "norman-ratio",
   "metadata": {},
   "source": [
    "因为您将自己限制在最常用的前 10,000 个单词中，所以没有单词索引会超过 10,000："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "curious-johnson",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9999"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 163 ms (started: 2021-07-22 15:17:33 +08:00)\n"
     ]
    }
   ],
   "source": [
    "max([max(sequence) for sequence in train_data])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "behavioral-rates",
   "metadata": {},
   "source": [
    "对于踢球，您可以通过以下方法将这些评论之一快速解码回英文单词："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "antique-rebecca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json\n",
      "1646592/1641221 [==============================] - 2s 1us/step\n",
      "time: 1.91 s (started: 2021-07-22 15:18:41 +08:00)\n"
     ]
    }
   ],
   "source": [
    "word_index = imdb.get_word_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "practical-adrian",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 56.3 ms (started: 2021-07-22 15:18:57 +08:00)\n"
     ]
    }
   ],
   "source": [
    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "young-correlation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 828 µs (started: 2021-07-22 15:21:47 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 解码评论。 请注意，索引偏移了 3，因为 0、1 和 2 是“填充”、“序列开始”和“未知”的保留索引。\n",
    "decoded_review = \" \".join([reverse_word_index.get(i - 3, \"?\") for i in train_data[0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "essential-rogers",
   "metadata": {},
   "source": [
    "### 准备数据"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proof-beach",
   "metadata": {},
   "source": [
    "你不能直接将整数列表输入神经网络。 它们的长度各不相同，但神经网络希望处理连续的批次数据。 你必须把你的列表变成张量。 有两种方法可以做到这一点："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "false-conditions",
   "metadata": {},
   "source": [
    "你不能直接将整数列表输入神经网络。 它们的长度各不相同，但神经网络希望处理连续的批次数据。 你必须把你的列表变成张量。 有两种方法可以做到这一点：\n",
    "* 填充您的列表，使它们都具有相同的长度，然后将它们转换为形状的整数张量（样本，max_length），并以能够处理此类整数张量的层（该层，我们将在其中介绍）开始您的模型 稍后在嵌入本书中详细介绍）。\n",
    "* 多热编码您的列表以将它们转换为 0 和 1 的向量。 这意味着，例如，将序列 [8, 5] 转换为 10,000 维向量，除了索引 8 和 5 为 1 外，该向量全为 0。 然后，您可以使用能够处理浮点矢量数据的 Dense 层作为模型中的第一层。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adult-april",
   "metadata": {},
   "source": [
    "让我们使用后一种解决方案来对数据进行矢量化，为了最大程度地清晰，您将手动执行此操作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "lonely-explosion",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 415 ms (started: 2021-07-23 14:25:12 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "def vectorize_sequences(sequences, dimension=10000):\n",
    "    \n",
    "    #创建一个全零矩阵的形状（len（序列），维度）\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        for j in sequence:\n",
    "            # 将结果 [i] 的特定索引设置为 1s\n",
    "            results[i, j] = 1.\n",
    "    return results\n",
    "\n",
    "# 矢量化训练数据\n",
    "x_train = vectorize_sequences(train_data)\n",
    "# 矢量化测试数据\n",
    "x_test = vectorize_sequences(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "historical-burton",
   "metadata": {},
   "source": [
    "以下是样本现在的样子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "boolean-offset",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 1., 1., ..., 0., 0., 0.])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.54 ms (started: 2021-07-22 15:30:17 +08:00)\n"
     ]
    }
   ],
   "source": [
    "x_train[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "robust-innocent",
   "metadata": {},
   "source": [
    "您还应该矢量化您的标签，这很简单："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "aging-pepper",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.42 ms (started: 2021-07-22 15:31:05 +08:00)\n"
     ]
    }
   ],
   "source": [
    "y_train = np.asarray(train_labels).astype(\"float32\")\n",
    "y_test = np.asarray(test_labels).astype(\"float32\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "certified-astronomy",
   "metadata": {},
   "source": [
    "现在数据已准备好输入神经网络。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "micro-analysis",
   "metadata": {},
   "source": [
    "### 构建模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "emerging-campbell",
   "metadata": {},
   "source": [
    "输入数据是向量，标签是标量（1 和 0）：这是您遇到的最简单的问题设置之一。 在此类问题上表现良好的一种模型是具有激活的密集连接 ( ) 层的普通堆栈。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "brutal-terror",
   "metadata": {},
   "source": [
    "输入数据是向量，标签是标量（1 和 0）：这是您遇到的最简单的问题设置之一。 在此类问题上表现良好的一种模型是具有 relu 激活的密集连接（dense）层的简单堆栈。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "close-yacht",
   "metadata": {},
   "source": [
    "关于这样的密集层堆栈，需要做出两个关键的架构决策：\n",
    "* 使用多少层\n",
    "* 每层选择多少个单位"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fuzzy-fifty",
   "metadata": {},
   "source": [
    "在第 5 章中，您将学习指导您做出这些选择的正式原则。 目前，您必须相信我的以下架构选择：\n",
    "* 两个中间层，每层 16 个单元\n",
    "* 第三层将输出关于当前评论情绪的标量预测"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "premier-reduction",
   "metadata": {},
   "source": [
    "图 4.1 显示了模型的外观。 这是 Keras 实现，类似于您之前看到的 MNIST 示例。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "authentic-happening",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/008i3skNly1gsps2310jnj30hi0pkq3v.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "adult-county",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.62 s (started: 2021-07-22 15:52:31 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "model = keras.Sequential([\n",
    "    layers.Dense(16, activation=\"relu\"),\n",
    "    layers.Dense(16, activation=\"relu\"),\n",
    "    layers.Dense(1, activation=\"sigmoid\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wound-procedure",
   "metadata": {},
   "source": [
    "传递给每一层的第一个参数是层中的数量：层表示空间的密集单位维数。 您还记得第 2 章和第 3 章中的每个具有激活的层都实现了以下张量密集 relu 操作链："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "iraqi-welding",
   "metadata": {},
   "outputs": [],
   "source": [
    "output = relu(dot(input, W) + b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "through-snake",
   "metadata": {},
   "source": [
    "有 16 个单位意味着权重矩阵将具有形状 W (input_dimension, 16)：点积与将输入数据投影到 16 维表示空间（然后 W 将添加偏置向量并应用操作）。 您可以直观地将表示空间的 b relu 维数理解为“在学习内部表示时，您允许模型拥有多少自由度”。 拥有更多单元（更高维的表示空间）可以让您的模型学习更复杂的表示，但它会使模型的计算成本更高，并可能导致学习不需要的模式（将提高训练数据性能但不会提高测试数据的性能） 测试数据）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "electric-controversy",
   "metadata": {},
   "source": [
    "中间层用作它们的激活函数，最后一层使用 sigmoid relu 激活以输出概率（0 到 1 之间的分数，表示样本有多少可能有目标“1”：评论的可能性有多大 是积极的）。 A（整流线性 relu 单元）是一个函数，旨在将负值归零（见图 4.2），而 sigmoid 将任意值“压缩”到区间中（见图 4.3），输出 [0, 1] 可以解释的东西 作为概率。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unavailable-summer",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/008i3skNly1gsps6hbncmj30n40req46.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "visible-pencil",
   "metadata": {},
   "source": [
    "> 什么是激活函数，为什么需要它们？"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "demographic-raising",
   "metadata": {},
   "source": [
    "如果没有像 relu（也称为 relu 非线性）这样的激活函数，密集层将包含两个线性运算——点积和加法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "interstate-ceremony",
   "metadata": {},
   "outputs": [],
   "source": [
    "output = dot(input, W) + b"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "basic-rotation",
   "metadata": {},
   "source": [
    "因此该层只能学习输入数据的线性变换（仿射变换）：该层的假设空间将是输入数据到 16 维空间的所有可能线性变换的集合。 这样的假设空间太受限制，不会从多层表示中受益，因为深度堆叠的线性层仍然会实现线性操作：添加更多层不会扩展假设空间（如您在第 2 章中看到的） ."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "elegant-guinea",
   "metadata": {},
   "source": [
    "为了获得更丰富的假设空间，\n",
    "从深度表示中受益，您需要非线性或激活函数。 relu是深度学习中最流行的激活函数，但是relu还有很多其他的候选函数，它们都有类似奇怪的名字：prelu、elu等等。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "superior-motorcycle",
   "metadata": {},
   "source": [
    "最后，您需要选择一个损失函数和一个优化器。 因为您面临二元分类问题，并且模型的输出是概率（您以具有 sigmoid 激活的单单元层结束模型），所以最好使用 binary_crossentropy 损失。 这不是唯一可行的选择：例如，您可以使用 mean_squared_error。 但是当您处理输出概率的模型时，交叉熵通常是最佳选择。 交叉熵是来自信息论领域的一个量，用于测量概率分布之间的距离，或者在这种情况下，真实分布与预测之间的距离。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "organized-worse",
   "metadata": {},
   "source": [
    "至于优化器的选择，我们将使用 rmsprop，对于几乎任何问题，这通常是一个很好的默认 rmsprop 选择。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hawaiian-metallic",
   "metadata": {},
   "source": [
    "这是使用 rmsprop 优化器和 binary_crossentropy 损失函数配置模型的步骤。 请注意，您还将在训练期间监控准确性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "naked-mixture",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 25.1 ms (started: 2021-07-22 16:01:22 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"binary_crossentropy\",\n",
    "              metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "corporate-checkout",
   "metadata": {},
   "source": [
    "### 验证您的方法"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ignored-visiting",
   "metadata": {},
   "source": [
    "正如您在第 3 章中了解到的，永远不应根据其训练数据评估深度学习模型——标准做法是在训练期间使用“验证集”来监控模型的准确性。 在这里，您将通过从原始训练数据中分离出 10,000 个样本来创建验证集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "increased-chinese",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 855 µs (started: 2021-07-22 16:50:45 +08:00)\n"
     ]
    }
   ],
   "source": [
    "x_val = x_train[:10000]\n",
    "partial_x_train = x_train[10000:]\n",
    "y_val = y_train[:10000]\n",
    "partial_y_train = y_train[10000:]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "arbitrary-pursuit",
   "metadata": {},
   "source": [
    "您现在将以 512 个样本的小批量训练模型 20 个时期（对训练数据中的所有样本进行 20 次迭代）。 同时，您将监控您分开的 10,000 个样本的损失和准确度。 您可以通过将验证数据作为validation_data 参数传递来实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "former-article",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "30/30 [==============================] - 5s 53ms/step - loss: 0.5269 - accuracy: 0.7865 - val_loss: 0.4028 - val_accuracy: 0.8706\n",
      "Epoch 2/20\n",
      "30/30 [==============================] - 1s 27ms/step - loss: 0.3201 - accuracy: 0.9016 - val_loss: 0.3389 - val_accuracy: 0.8661\n",
      "Epoch 3/20\n",
      "30/30 [==============================] - 1s 28ms/step - loss: 0.2304 - accuracy: 0.9266 - val_loss: 0.2801 - val_accuracy: 0.8920\n",
      "Epoch 4/20\n",
      "30/30 [==============================] - 1s 25ms/step - loss: 0.1812 - accuracy: 0.9407 - val_loss: 0.2883 - val_accuracy: 0.8838\n",
      "Epoch 5/20\n",
      "30/30 [==============================] - 1s 26ms/step - loss: 0.1488 - accuracy: 0.9508 - val_loss: 0.2799 - val_accuracy: 0.8872\n",
      "Epoch 6/20\n",
      "30/30 [==============================] - 1s 26ms/step - loss: 0.1238 - accuracy: 0.9616 - val_loss: 0.2994 - val_accuracy: 0.8824\n",
      "Epoch 7/20\n",
      "30/30 [==============================] - 1s 28ms/step - loss: 0.1016 - accuracy: 0.9701 - val_loss: 0.3057 - val_accuracy: 0.8825\n",
      "Epoch 8/20\n",
      "30/30 [==============================] - 1s 33ms/step - loss: 0.0867 - accuracy: 0.9737 - val_loss: 0.3261 - val_accuracy: 0.8809\n",
      "Epoch 9/20\n",
      "30/30 [==============================] - 1s 29ms/step - loss: 0.0696 - accuracy: 0.9805 - val_loss: 0.3576 - val_accuracy: 0.8775\n",
      "Epoch 10/20\n",
      "30/30 [==============================] - 1s 30ms/step - loss: 0.0593 - accuracy: 0.9845 - val_loss: 0.3746 - val_accuracy: 0.8778\n",
      "Epoch 11/20\n",
      "30/30 [==============================] - 1s 28ms/step - loss: 0.0484 - accuracy: 0.9872 - val_loss: 0.3965 - val_accuracy: 0.8779\n",
      "Epoch 12/20\n",
      "30/30 [==============================] - 1s 28ms/step - loss: 0.0405 - accuracy: 0.9902 - val_loss: 0.4314 - val_accuracy: 0.8756\n",
      "Epoch 13/20\n",
      "30/30 [==============================] - 1s 28ms/step - loss: 0.0314 - accuracy: 0.9932 - val_loss: 0.4697 - val_accuracy: 0.8747\n",
      "Epoch 14/20\n",
      "30/30 [==============================] - 1s 30ms/step - loss: 0.0283 - accuracy: 0.9936 - val_loss: 0.4857 - val_accuracy: 0.8736\n",
      "Epoch 15/20\n",
      "30/30 [==============================] - 1s 27ms/step - loss: 0.0201 - accuracy: 0.9968 - val_loss: 0.5159 - val_accuracy: 0.8705\n",
      "Epoch 16/20\n",
      "30/30 [==============================] - 1s 28ms/step - loss: 0.0161 - accuracy: 0.9975 - val_loss: 0.5533 - val_accuracy: 0.8714\n",
      "Epoch 17/20\n",
      "30/30 [==============================] - 1s 29ms/step - loss: 0.0152 - accuracy: 0.9967 - val_loss: 0.5912 - val_accuracy: 0.8692\n",
      "Epoch 18/20\n",
      "30/30 [==============================] - 1s 25ms/step - loss: 0.0104 - accuracy: 0.9985 - val_loss: 0.6199 - val_accuracy: 0.8676\n",
      "Epoch 19/20\n",
      "30/30 [==============================] - 1s 24ms/step - loss: 0.0081 - accuracy: 0.9989 - val_loss: 0.6577 - val_accuracy: 0.8673\n",
      "Epoch 20/20\n",
      "30/30 [==============================] - 1s 28ms/step - loss: 0.0049 - accuracy: 0.9999 - val_loss: 0.7015 - val_accuracy: 0.8670\n",
      "time: 21.8 s (started: 2021-07-22 16:51:39 +08:00)\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(partial_x_train,\n",
    "                    partial_y_train,\n",
    "                    epochs=20,\n",
    "                    batch_size=512,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "noble-entertainment",
   "metadata": {},
   "source": [
    "在 CPU 上，这将花费每个 epoch 不到 2 秒的时间——训练在 20 秒内结束。 在每个 epoch 结束时，模型会在 10,000 个验证数据样本上计算其损失和准确度，因此会有轻微的停顿。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "searching-dispute",
   "metadata": {},
   "source": [
    "请注意，调用 model.fit() 返回一个 History 对象，正如您在第 3 章中看到的那样。该对象有一个成员 ，它是一个字典，其中包含有关训练期间发生的所有历史数据的数据。 让我们来看看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "going-retreat",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 453 µs (started: 2021-07-22 16:54:56 +08:00)\n"
     ]
    }
   ],
   "source": [
    "history_dict = history.history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "accomplished-identity",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.38 ms (started: 2021-07-22 16:55:08 +08:00)\n"
     ]
    }
   ],
   "source": [
    "history_dict.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "bronze-taste",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 461 µs (started: 2021-07-22 16:55:41 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "empirical-karma",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'loss': [0.526863694190979,\n",
       "  0.320054292678833,\n",
       "  0.23038016259670258,\n",
       "  0.18120521306991577,\n",
       "  0.14881940186023712,\n",
       "  0.12377016246318817,\n",
       "  0.10163230448961258,\n",
       "  0.08668980002403259,\n",
       "  0.06957581639289856,\n",
       "  0.05931144952774048,\n",
       "  0.04841609671711922,\n",
       "  0.040512919425964355,\n",
       "  0.03139404579997063,\n",
       "  0.028269965201616287,\n",
       "  0.020131245255470276,\n",
       "  0.01613525114953518,\n",
       "  0.015231958590447903,\n",
       "  0.010360080748796463,\n",
       "  0.008133160881698132,\n",
       "  0.0048561799339950085],\n",
       " 'accuracy': [0.7865333557128906,\n",
       "  0.9016000032424927,\n",
       "  0.9265999794006348,\n",
       "  0.9407333135604858,\n",
       "  0.9508000016212463,\n",
       "  0.9616000056266785,\n",
       "  0.9701333045959473,\n",
       "  0.9737333059310913,\n",
       "  0.9804666638374329,\n",
       "  0.9844666719436646,\n",
       "  0.9872000217437744,\n",
       "  0.9901999831199646,\n",
       "  0.9932000041007996,\n",
       "  0.9936000108718872,\n",
       "  0.9968000054359436,\n",
       "  0.9974666833877563,\n",
       "  0.9967333078384399,\n",
       "  0.998533308506012,\n",
       "  0.9988666772842407,\n",
       "  0.9998666644096375],\n",
       " 'val_loss': [0.4028334617614746,\n",
       "  0.338904470205307,\n",
       "  0.28012749552726746,\n",
       "  0.2882697582244873,\n",
       "  0.27992376685142517,\n",
       "  0.299434632062912,\n",
       "  0.3056628704071045,\n",
       "  0.32607725262641907,\n",
       "  0.3576413094997406,\n",
       "  0.3746439218521118,\n",
       "  0.39645692706108093,\n",
       "  0.4313766658306122,\n",
       "  0.46967315673828125,\n",
       "  0.48573145270347595,\n",
       "  0.5158823132514954,\n",
       "  0.5533170700073242,\n",
       "  0.5912437438964844,\n",
       "  0.6199294924736023,\n",
       "  0.6577344536781311,\n",
       "  0.7014950513839722],\n",
       " 'val_accuracy': [0.8705999851226807,\n",
       "  0.866100013256073,\n",
       "  0.8920000195503235,\n",
       "  0.8838000297546387,\n",
       "  0.8871999979019165,\n",
       "  0.8823999762535095,\n",
       "  0.8824999928474426,\n",
       "  0.8809000253677368,\n",
       "  0.8774999976158142,\n",
       "  0.8777999877929688,\n",
       "  0.8779000043869019,\n",
       "  0.8755999803543091,\n",
       "  0.8747000098228455,\n",
       "  0.8736000061035156,\n",
       "  0.8705000281333923,\n",
       "  0.871399998664856,\n",
       "  0.8691999912261963,\n",
       "  0.8676000237464905,\n",
       "  0.8672999739646912,\n",
       "  0.8669999837875366]}"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.48 ms (started: 2021-07-22 16:56:40 +08:00)\n"
     ]
    }
   ],
   "source": [
    "history.history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "waiting-anthony",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>val_loss</th>\n",
       "      <th>val_accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.526864</td>\n",
       "      <td>0.786533</td>\n",
       "      <td>0.402833</td>\n",
       "      <td>0.8706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.320054</td>\n",
       "      <td>0.901600</td>\n",
       "      <td>0.338904</td>\n",
       "      <td>0.8661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.230380</td>\n",
       "      <td>0.926600</td>\n",
       "      <td>0.280127</td>\n",
       "      <td>0.8920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.181205</td>\n",
       "      <td>0.940733</td>\n",
       "      <td>0.288270</td>\n",
       "      <td>0.8838</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.148819</td>\n",
       "      <td>0.950800</td>\n",
       "      <td>0.279924</td>\n",
       "      <td>0.8872</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.123770</td>\n",
       "      <td>0.961600</td>\n",
       "      <td>0.299435</td>\n",
       "      <td>0.8824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.101632</td>\n",
       "      <td>0.970133</td>\n",
       "      <td>0.305663</td>\n",
       "      <td>0.8825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.086690</td>\n",
       "      <td>0.973733</td>\n",
       "      <td>0.326077</td>\n",
       "      <td>0.8809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.069576</td>\n",
       "      <td>0.980467</td>\n",
       "      <td>0.357641</td>\n",
       "      <td>0.8775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.059311</td>\n",
       "      <td>0.984467</td>\n",
       "      <td>0.374644</td>\n",
       "      <td>0.8778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.048416</td>\n",
       "      <td>0.987200</td>\n",
       "      <td>0.396457</td>\n",
       "      <td>0.8779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.040513</td>\n",
       "      <td>0.990200</td>\n",
       "      <td>0.431377</td>\n",
       "      <td>0.8756</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.031394</td>\n",
       "      <td>0.993200</td>\n",
       "      <td>0.469673</td>\n",
       "      <td>0.8747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.028270</td>\n",
       "      <td>0.993600</td>\n",
       "      <td>0.485731</td>\n",
       "      <td>0.8736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.020131</td>\n",
       "      <td>0.996800</td>\n",
       "      <td>0.515882</td>\n",
       "      <td>0.8705</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.016135</td>\n",
       "      <td>0.997467</td>\n",
       "      <td>0.553317</td>\n",
       "      <td>0.8714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.015232</td>\n",
       "      <td>0.996733</td>\n",
       "      <td>0.591244</td>\n",
       "      <td>0.8692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.010360</td>\n",
       "      <td>0.998533</td>\n",
       "      <td>0.619929</td>\n",
       "      <td>0.8676</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0.008133</td>\n",
       "      <td>0.998867</td>\n",
       "      <td>0.657734</td>\n",
       "      <td>0.8673</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0.004856</td>\n",
       "      <td>0.999867</td>\n",
       "      <td>0.701495</td>\n",
       "      <td>0.8670</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        loss  accuracy  val_loss  val_accuracy\n",
       "0   0.526864  0.786533  0.402833        0.8706\n",
       "1   0.320054  0.901600  0.338904        0.8661\n",
       "2   0.230380  0.926600  0.280127        0.8920\n",
       "3   0.181205  0.940733  0.288270        0.8838\n",
       "4   0.148819  0.950800  0.279924        0.8872\n",
       "5   0.123770  0.961600  0.299435        0.8824\n",
       "6   0.101632  0.970133  0.305663        0.8825\n",
       "7   0.086690  0.973733  0.326077        0.8809\n",
       "8   0.069576  0.980467  0.357641        0.8775\n",
       "9   0.059311  0.984467  0.374644        0.8778\n",
       "10  0.048416  0.987200  0.396457        0.8779\n",
       "11  0.040513  0.990200  0.431377        0.8756\n",
       "12  0.031394  0.993200  0.469673        0.8747\n",
       "13  0.028270  0.993600  0.485731        0.8736\n",
       "14  0.020131  0.996800  0.515882        0.8705\n",
       "15  0.016135  0.997467  0.553317        0.8714\n",
       "16  0.015232  0.996733  0.591244        0.8692\n",
       "17  0.010360  0.998533  0.619929        0.8676\n",
       "18  0.008133  0.998867  0.657734        0.8673\n",
       "19  0.004856  0.999867  0.701495        0.8670"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 13.9 ms (started: 2021-07-22 16:56:50 +08:00)\n"
     ]
    }
   ],
   "source": [
    "pd.DataFrame(history.history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "regional-delivery",
   "metadata": {},
   "source": [
    "该字典包含四个条目：每个在训练和验证期间被监控的指标一个。 在以下两个清单中，让我们使用 Matplotlib 并排绘制训练和验证损失（见图 4.4），以及训练和验证准确度（见图 4.5）。 请注意，由于模型的随机初始化不同，您自己的结果可能会略有不同。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "persistent-cooperation",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAyw0lEQVR4nO3deXhU5fn/8fctILuogBu7ClJUCBBQpFBcWgEtKi6AVMENsVqXtlosVflqaW3lZ60tanFf0ga1rUUFURGKS1UCKhVERQQNdUEsWwEhcP/+eE5gCJON5GQmM5/Xdc2VmTNnztw5mZx7nt3cHRERyV57pToAERFJLSUCEZEsp0QgIpLllAhERLKcEoGISJZTIhARyXJKBFKtzGyGmY2q7n1TycyWm9lJMRzXzezw6P49ZnZDRfbdg/cZaWbP72mcZRx3gJkVVvdxpebVTXUAknpmtiHhYSPgG2Bb9PhSd8+r6LHcfVAc+2Y6dx9bHccxs/bAx0A9dy+Kjp0HVPhvKNlHiUBw9ybF981sOXCxu79Ycj8zq1t8cRGRzKGqISlVcdHfzH5mZp8DD5rZfmb2jJmtMrP/RvdbJ7xmjpldHN0fbWavmNmkaN+PzWzQHu7bwczmmtl6M3vRzCab2WOlxF2RGG8xs1ej4z1vZi0Snj/PzFaY2WozG1/G+TnGzD43szoJ284ws4XR/d5m9i8zW2Nmn5nZH81s71KO9ZCZ/TLh8bXRa/5jZheW2PcUM3vLzNaZ2admNiHh6bnRzzVmtsHM+hSf24TXH2dm88xsbfTzuIqem7KY2bei168xs0VmNiThucFmtjg65koz+2m0vUX091ljZl+b2ctmputSDdMJl/IcBOwPtAPGED4zD0aP2wKbgD+W8fpjgPeBFsBvgfvNzPZg3z8DbwLNgQnAeWW8Z0ViPBe4ADgA2BsovjB1Ae6Ojn9I9H6tScLd3wD+B5xQ4rh/ju5vA66Jfp8+wInAD8uImyiGgVE83wU6AiXbJ/4HnA/sC5wCXGZmp0fP9Y9+7uvuTdz9XyWOvT/wLHBn9LvdDjxrZs1L/A67nZtyYq4HPA08H73uR0CemR0R7XI/oZqxKXAU8FK0/SdAIdASOBD4OaB5b2qYEoGUZztwk7t/4+6b3H21u//V3Te6+3pgIvCdMl6/wt3vdfdtwMPAwYR/+Arva2ZtgV7Aje6+xd1fAaaV9oYVjPFBd//A3TcBjwM50fazgGfcfa67fwPcEJ2D0vwFGAFgZk2BwdE23H2+u7/u7kXuvhz4U5I4kjkniu9dd/8fIfEl/n5z3P3f7r7d3RdG71eR40JIHB+6+6NRXH8BlgDfT9intHNTlmOBJsCt0d/oJeAZonMDbAW6mNk+7v5fd1+QsP1goJ27b3X3l10ToNU4JQIpzyp331z8wMwamdmfoqqTdYSqiH0Tq0dK+Lz4jrtvjO42qeS+hwBfJ2wD+LS0gCsY4+cJ9zcmxHRI4rGjC/Hq0t6L8O1/qJnVB4YCC9x9RRRHp6ja4/Mojl8RSgfl2SUGYEWJ3+8YM5sdVX2tBcZW8LjFx15RYtsKoFXC49LOTbkxu3ti0kw87pmEJLnCzP5pZn2i7bcBS4HnzWyZmY2r2K8h1UmJQMpT8tvZT4AjgGPcfR92VkWUVt1THT4D9jezRgnb2pSxf1Vi/Czx2NF7Ni9tZ3dfTLjgDWLXaiEIVUxLgI5RHD/fkxgI1VuJ/kwoEbVx92bAPQnHLe/b9H8IVWaJ2gIrKxBXecdtU6J+f8dx3X2eu59GqDZ6ilDSwN3Xu/tP3P1QYAjwYzM7sYqxSCUpEUhlNSXUua+J6ptvivsNo2/YBcAEM9s7+jb5/TJeUpUYnwRONbNvRw27N1P+/8mfgasICeeJEnGsAzaYWWfgsgrG8Dgw2sy6RImoZPxNCSWkzWbWm5CAiq0iVGUdWsqxpwOdzOxcM6trZsOALoRqnKp4g1B6uM7M6pnZAMLfKD/6m400s2buvpVwTrYDmNmpZnZ41Ba0ltCuUlZVnMRAiUAq6w6gIfAV8DrwXA2970hCg+tq4JfAVMJ4h2TuYA9jdPdFwOWEi/tnwH8JjZllKa6jf8ndv0rY/lPCRXo9cG8Uc0VimBH9Di8Rqk1eKrHLD4GbzWw9cCPRt+votRsJbSKvRj1xji1x7NXAqYRS02rgOuDUEnFXmrtvIVz4BxHO+13A+e6+JNrlPGB5VEU2lvD3hNAY/iKwAfgXcJe7z65KLFJ5pnYZqY3MbCqwxN1jL5GIZDqVCKRWMLNeZnaYme0Vda88jVDXLCJVpJHFUlscBPyN0HBbCFzm7m+lNiSRzKCqIRGRLKeqIRGRLFfrqoZatGjh7du3T3UYIiK1yvz5879y95bJnqt1iaB9+/YUFBSkOgwRkVrFzEqOKN9BVUMiIllOiUBEJMspEYiIZLla10aQzNatWyksLGTz5s3l7ywp1aBBA1q3bk29evVSHYqIRDIiERQWFtK0aVPat29P6WueSKq5O6tXr6awsJAOHTqkOhwRiWRE1dDmzZtp3ry5kkCaMzOaN2+ukptImok1EZjZQDN738yWJltwwsx+Z2ZvR7cPzGxNFd6rSrFKzdDfSST9xJYIotWgJhOmpe0CjIjWg93B3a9x9xx3zwH+QJhLRkREEnzzDVx7LXxa6rp8VRNniaA3sNTdl0VzlecTZowszQiitV5rm9WrV5OTk0NOTg4HHXQQrVq12vF4y5YtZb62oKCAK6+8stz3OO6446ol1jlz5nDqqadWy7FEJH6ffAL9+sGkSfDss/G8R5yNxa3Ydd3VQuCYZDuaWTugA7svwFH8/BhgDEDbtiVX7au8vDwYPz6c4LZtYeJEGDmy/NeVpnnz5rz99tsATJgwgSZNmvDTn/50x/NFRUXUrZv8VOfm5pKbm1vue7z22mt7HqCI1ErPPw/nngtbtsDf/gZnnBHP+6RLY/Fw4El335bsSXef4u657p7bsmXSqTIqLC8PxoyBFSvAPfwcMyZsr06jR49m7NixHHPMMVx33XW8+eab9OnTh+7du3Pcccfx/vvvA7t+Q58wYQIXXnghAwYM4NBDD+XOO+/ccbwmTZrs2H/AgAGcddZZdO7cmZEjR1I8g+z06dPp3LkzPXv25Morryz3m//XX3/N6aefTteuXTn22GNZuHAhAP/85z93lGi6d+/O+vXr+eyzz+jfvz85OTkcddRRvPzyy9V7wkRkh+3b4ZZbYOBAOPhgKCiILwlAvCWCley6AHdrSl8gezhhecDYjR8PGzfuum3jxrC9KqWCZAoLC3nttdeoU6cO69at4+WXX6Zu3bq8+OKL/PznP+evf/3rbq9ZsmQJs2fPZv369RxxxBFcdtllu/W5f+utt1i0aBGHHHIIffv25dVXXyU3N5dLL72UuXPn0qFDB0aMGFFufDfddBPdu3fnqaee4qWXXuL888/n7bffZtKkSUyePJm+ffuyYcMGGjRowJQpUzj55JMZP34827ZtY2PJkygi1eLrr+G882D6dPjBD+Cee6Bx43jfM85EMA/oaGYdCAlgOLsusg1AtKj3foT1SmP3ySeV214VZ599NnXq1AFg7dq1jBo1ig8//BAzY+vWrUlfc8opp1C/fn3q16/PAQccwBdffEHr1q132ad37947tuXk5LB8+XKaNGnCoYceuqN//ogRI5gyZUqZ8b3yyis7ktEJJ5zA6tWrWbduHX379uXHP/4xI0eOZOjQobRu3ZpevXpx4YUXsnXrVk4//XRycnKqcmpEJIkFC+DMM2HlSrjrLhg7Fmqio11sVUPuXgRcAcwE3gMed/dFZnazmQ1J2HU4kO81tEJOaU0M1dD0sJvGCWn8hhtu4Pjjj+fdd9/l6aefLrUvff369Xfcr1OnDkVFRXu0T1WMGzeO++67j02bNtG3b1+WLFlC//79mTt3Lq1atWL06NE88sgj1fqeItnu/vvhuOOgqAhefhkuu6xmkgDE3Ebg7tPdvZO7H+buE6NtN7r7tIR9Jrj7bmMM4jJxIjRqtOu2Ro3C9jitXbuWVq1aAfDQQw9V+/GPOOIIli1bxvLlywGYOnVqua/p168feVHjyJw5c2jRogX77LMPH330EUcffTQ/+9nP6NWrF0uWLGHFihUceOCBXHLJJVx88cUsWLCg2n8HkWy0aRNcdBFcfHHoHbRgARyTtFtNfNKlsbjGjBwJU6ZAu3Yh27ZrFx5Xd/tASddddx3XX3893bt3r/Zv8AANGzbkrrvuYuDAgfTs2ZOmTZvSrFmzMl8zYcIE5s+fT9euXRk3bhwPP/wwAHfccQdHHXUUXbt2pV69egwaNIg5c+bQrVs3unfvztSpU7nqqquq/XcQyTbLlkHfvvDAA6Gd8rnnoIr9YfZIrVuzODc310suTPPee+/xrW99K0URpY8NGzbQpEkT3J3LL7+cjh07cs0116Q6rN3o7yUCzzwTGoUBHn0U4h7eY2bz3T1pX/WsKxFksnvvvZecnByOPPJI1q5dy6WXXprqkESkhG3b4IYb4Pvfh/btYf78+JNAeTJi9lEJrrnmmrQsAYhI8NVXYYDYCy/ABRfA5MnQsGGqo1IiEBGpEW+8AWefDV9+CffeGxqH04WqhkREYrR1K/zyl6FHUJ068Oqr6ZUEQCUCEZHYvPsujBoVuoQOGxYGie2/f6qj2p1KBCIi1ayoCH71K+jRI0wd/eSTkJ+fnkkAlAiqxfHHH8/MmTN32XbHHXdw2WWXlfqaAQMGUNwNdvDgwaxZs2a3fSZMmMCkSZPKfO+nnnqKxYsX73h844038uKLL1Yi+uQ0XbXInlm8GPr0CeMCTj8dFi0K00akMyWCajBixAjy8/N32Zafn1+hid8gzBq677777tF7l0wEN998MyeddNIeHUtE9lxREdx6K3TvDsuXw9Sp8PjjqRkgVllKBNXgrLPO4tlnn92xCM3y5cv5z3/+Q79+/bjsssvIzc3lyCOP5Kabbkr6+vbt2/PVV18BMHHiRDp16sS3v/3tHVNVQxgj0KtXL7p168aZZ57Jxo0bee2115g2bRrXXnstOTk5fPTRR4wePZonn3wSgFmzZtG9e3eOPvpoLrzwQr755psd73fTTTfRo0cPjj76aJYsWVLm76fpqkXK9t57YYTw9deH8QGLFsE556Q6qorLuMbiq6+GaI2YapOTA3fcUfrz+++/P71792bGjBmcdtpp5Ofnc84552BmTJw4kf33359t27Zx4oknsnDhQrp27Zr0OPPnzyc/P5+3336boqIievToQc+ePQEYOnQol1xyCQC/+MUvuP/++/nRj37EkCFDOPXUUznrrLN2OdbmzZsZPXo0s2bNolOnTpx//vncfffdXH311QC0aNGCBQsWcNdddzFp0iTuu+++Un8/TVctkty2bXD77WGAWOPG8Je/hEbh2rY0t0oE1SSxeiixWujxxx+nR48edO/enUWLFu1SjVPSyy+/zBlnnEGjRo3YZ599GDJk5ySt7777Lv369ePoo48mLy+PRYsWlRnP+++/T4cOHejUqRMAo0aNYu7cuTueHzp0KAA9e/bcMVFdaV555RXOi8bCJ5uu+s4772TNmjXUrVuXXr168eCDDzJhwgT+/e9/07Rp0zKPLVJbLVkC3/42XHcdDB4c2gaGD699SQAysERQ1jf3OJ122mlcc801LFiwgI0bN9KzZ08+/vhjJk2axLx589hvv/0YPXp0qdNPl2f06NE89dRTdOvWjYceeog5c+ZUKd7iqayrMo31uHHjOOWUU5g+fTp9+/Zl5syZO6arfvbZZxk9ejQ//vGPOf/886sUq0g62bYtXGfGjw8zF+flwYgRtTMBFFOJoJo0adKE448/ngsvvHBHaWDdunU0btyYZs2a8cUXXzBjxowyj9G/f3+eeuopNm3axPr163n66ad3PLd+/XoOPvhgtm7dumPqaICmTZuyfv363Y51xBFHsHz5cpYuXQrAo48+yne+8509+t00XbVI8MEH0L8//PSncPLJoS3g3HNrdxKADCwRpNKIESM444wzdlQRFU/b3LlzZ9q0aUPfvn3LfH2PHj0YNmwY3bp144ADDqBXr147nrvllls45phjaNmyJcccc8yOi//w4cO55JJLuPPOO3c0EgM0aNCABx98kLPPPpuioiJ69erF2LFj9+j3Kl5LuWvXrjRq1GiX6apnz57NXnvtxZFHHsmgQYPIz8/ntttuo169ejRp0kQL2EhG2LIF/vjHUApo2DDMFjpyZO1PAMU0DbXUOP29pLYoKoJHHgkLyS9fHmYJ/dOf4JBDUh1Z5WkaahGRSigqCt/6O3cOq4c1bx7WD5g2rXYmgfIoEYiIRLZtC11AjzoKzj8fmjSBf/wD5s2DU07JnKqgkmJNBGY20MzeN7OlZpZ0XWIzO8fMFpvZIjP7856+V22r4spW+jtJOtq+HZ54Arp2DY2/9erBX/8aJosbMiRzE0Cx2BKBmdUBJgODgC7ACDPrUmKfjsD1QF93PxK4ek/eq0GDBqxevVoXmTTn7qxevZoGDRqkOhQRANzh738Pg0bPOSc8njoV3nkHhg6FvbKkziTOXkO9gaXuvgzAzPKB04DEEVWXAJPd/b8A7v7lnrxR69atKSwsZNWqVVUMWeLWoEEDWrduneowJMu5hzr/m26Ct96Cjh3DeIBhw8KaAdkmzkTQCvg04XEhcEyJfToBmNmrQB1ggrs/V/JAZjYGGAPQtm3b3d6oXr16dOjQoXqiFpGM5Q7PPQc33ggFBXDoofDQQ6EraN0s7kyf6oJPXaAjMAAYAdxrZvuW3Mndp7h7rrvntqwNU/mJSNqZNStMDDd4MKxaBffdF6aJGDUqu5MAxJsIVgJtEh63jrYlKgSmuftWd/8Y+ICQGEREqsXXX8MPfgAnnRQWibnnnjBC+KKLQqOwxJsI5gEdzayDme0NDAemldjnKUJpADNrQagqWhZjTCKSRZ55Bo48MjQA33QTLF0Kl14Ke++d6sjSS2yJwN2LgCuAmcB7wOPuvsjMbjaz4mk1ZwKrzWwxMBu41t1XxxWTiGSHNWtg9OiwNkDLlvDmmzBhAkRzLUoJGTHFhIhIsRkz4OKL4YsvwkIxN9ygEgBoigkRyQJr14YEMHgw7LsvvP56mCNISaB8SgQiUuu98AIcfTQ8+CCMGxdGBOcm/e4rySgRiEittX49jB0L3/teWCrytdfg179WW0BlKRGISK300kuhFDBlSlgoZsECOKbkkFWpECUCEalVNmyAK66AE08M9f+vvAK33RYWjJE9o0QgIrXG3LnQrRvcdRdcfTW8/TYcd1yqo6r9lAhEJO3973/hwl+87PacOfC734XF46XqsnyGDRFJd3PnwoUXwkcfhSqhW28NDcNSfVQiEJG0tGED/OhHoRTgDrNnwx/+oCQQByUCEUk7s2eH1cL++Ee48kpYuBAGDEh1VJlLiUBE0saGDXD55XDCCWGBmLlz4fe/VykgbkoEIpIWZs0Ki8bffTdcc01YLrJfv1RHlR2yIhHk5UH79mH90fbtw2MRSQ/r1oXRwSedFMYFvPwy3H67egTVpIzvNZSXB2PGwMaN4fGKFeExhOXpRCR1nn8+TBRXWAg/+UmYJE4Dw2pexpcIxo/fmQSKbdwYtotIaqxdC5dcAiefHOr/X30VJk1SEkiVjE8En3xSue0iEq/nngttAQ88AD/7Gbz1FvTpk+qoslvGJ4K2bSu3XUTisWZNGBg2aBDssw/8619hcFiDBqmOTDI+EUycuHujU6NGYbuIxG/bNnjoobB28COPhFXD5s+H3r1THZkUy/hEMHJkmKa2XTswCz+nTFFDsUjc3OEf/wgDwy64AA45JKwa9qtfqRSQbmJNBGY20MzeN7OlZjYuyfOjzWyVmb0d3S6OI46RI2H5cti+PfxUEhCJ19y50LcvnH46FBXBE0+EBeS1alh6iq37qJnVASYD3wUKgXlmNs3dF5fYdaq7XxFXHCJScxYuDFU/06eHEsCUKaE0UDfjO6rXbnGWCHoDS919mbtvAfKB02J8PxFJkY8/hvPOg5ycsFzkrbfChx+GLqJKAukvzkTQCvg04XFhtK2kM81soZk9aWZtkh3IzMaYWYGZFaxatSqOWEVkD3z5ZZgU7ogj4Mkn4brrYNmy0C1UI4Nrj1Q3Fj8NtHf3rsALwMPJdnL3Ke6e6+65LVu2rNEARWR369bBhAlw2GFhtbALLoClS0NJYL/9Uh2dVFaciWAlkPgNv3W0bQd3X+3u30QP7wN6xhiPiFTRN9+E2UAPOwz+7/9g4EBYtAj+9Cdolay8L7VCnIlgHtDRzDqY2d7AcGBa4g5mdnDCwyHAezHGIyJ7aMMGePhh6Nw5LBnZtWvoBfTEE6FaSGq32Jpx3L3IzK4AZgJ1gAfcfZGZ3QwUuPs04EozGwIUAV8Do+OKR0Qqzh0WLw7TQcyYEWYE3bIFevQIPYFOOimMy5HMYO6e6hgqJTc31wsKClIdhkjGWb8+rAkwY0ZIAMXzcR11VKgCGjw4LBu5V6pbFmWPmNl8d086kkMdu0SylDu8+2648M+YAa+8EgZ/NW0avvH/4hchAbRJ2pdPMokSgUgWWbsWXnxx57f+lVH3ja5dw3oAgwaFmUD33ju1cUrNUiIQyQJvvgnjxoW6/qIiaNYMvvvd8I1/4ED1+Ml2SgQiGWzLFrj55tC//6CD4Nprw7f+Y4+FevVSHZ2kCyUCkQz1zjswalT4ecEF8LvfhZKASElq/xfJMEVFYb2NXr3g889h2rSwGpiSgJRGJQKRDLJkCZx/PsybB8OGweTJ0Lx5qqOSdKcSgUgG2L49VP107x4mfZs6FfLzlQSkYlQiEKnlPvootAG8/DIMGRLm/TnooFRHJbWJSgQitZQ73HMPdOsWFoR5+GF46iklAak8lQhEaqFPP4WLLoIXXgjjAe6/XyOAZc+pRCBSi7iHb/5HHRVWArvnHpg5U0lAqkYlApFa4vPPYcwYePpp6N8fHnwQDj001VFJJlCJQCTNJZYCXngh9A6aPVtJQKqPEoFIGnvvPTj+eBg9OiwA89ZbYWEYTQUt1UkfJ5E0tGlTmAa6uEfQvfeG7qGdO6c6MslEaiMQSTPPPQeXXx4Gho0aBbfdBi1bpjoqyWQqEYikif/8J0wLMWhQmBl09mx46CElAYmfEoFIim3bBn/4Q6j2+cc/4JZbwoyhAwakOjLJFqoaEkmh+fPh0kvDz+99L0wSd/jhqY5Ksk2sJQIzG2hm75vZUjMbV8Z+Z5qZm1nShZWry/btcR5dpOLWroUrr4TevcNykfn5oW1ASUBSIbZEYGZ1gMnAIKALMMLMuiTZrylwFfBGXLEA/O1v0LcvrFsX57uIlM0dnngCvvUt+OMf4Yc/DFNHDxsGZqmOTrJVnCWC3sBSd1/m7luAfOC0JPvdAvwG2BxjLDRuHOZoP+ss2Lo1zncSSW7ZMhg8GM45J0wM98YboW1AC8ZIqsWZCFoBnyY8Loy27WBmPYA27v5sWQcyszFmVmBmBatWrdqjYE4+OfTFfuEFuPji8M1MJG7uMHdu6AbapQu88grccUdYTL5Xr1RHJxKkrLHYzPYCbgdGl7evu08BpgDk5ubu8SX8ggvCrI033QRt24beGSJx+OyzMC3EAw/Ahx9C06ZhdPANN0CrVuW+XKRGxZkIVgKJcyK2jrYVawocBcyxUDl6EDDNzIa4e0FcQd1wA3zyCfzyl2HGxjFj4nonyTZbt8Kzz4YpoWfMCN1C+/WD8eNDlWTjxqmOUCS5OBPBPKCjmXUgJIDhwLnFT7r7WqBF8WMzmwP8NM4kEN4H7r47DN657DI45BA49dQ431Ey3fvvh4v/I4/AF1+E+v9rrw0l0E6dUh2dSPliSwTuXmRmVwAzgTrAA+6+yMxuBgrcfVpc712eevXg8cfDgJ1hw2DOHNXXSuVs2BB6/9x/P7z6KtSpE75QXHRRGBlcVyN0pBYxr0CrqZk1Bja5+3Yz6wR0Bma4e433v8nNzfWCguopNHzxBfTpE/6p//UvOOywajmsZCh3eP31UO+fnx8+N506hYv/+edriUhJb2Y2392TjtWq6PeWuUA/M9sPeJ5Q7TMMGFk9IabGgQeGutzjjgvf4l57DVq0KP91kj22bw+jfp9+Gv76V1i8GBo1Cl1AL7oojE1R/3+p7SqaCMzdN5rZRcBd7v5bM3s7xrhqzBFHhH/yE0+E738fZs0K/+iSvTZuhBdfDJ+LZ54JK4PttVf4wjBlSqhO3GefVEcpUn0qnAjMrA+hBHBRtK1OPCHVvOOOg7y80LNj5Eh48slQ5yvZY+XKcNF/+unwZWDz5tDlc+DA8AVh0CCVFiVzVTQRXA1cD/w9avA9FJgdW1QpMHQo/P73Yf6Xq64KIz5V5M9c7mG1r6efhmnTYMGCsL19e7jkknDx/853YO+9UxqmSI2oUCJw938C/4QdA8G+cvcr4wwsFX70ozDGYNIkaNcudAGUzLFpE7z00s4qn5UrQ7I/9lj41a/Cxf/II/UFQLJPhRKBmf0ZGAtsIzQU72Nmv3f32+IMLhV+85sw+vi666B1axgxItURSVWtXw+33hpKfP/7XxjYdfLJ4cI/eDAccECqIxRJrYpWDXVx93VmNhKYAYwD5gMZlwj22itMDfD552F+mIMOCouHS+1TVBT6+d94I3z5JQwfHqZ5GDAA6tdPdXQi6aOik87VM7N6wOnAtGj8QMZO21a/Pvz979CxI5xxBrz7bqojksp67jnIyYGxY0PPsDffhL/8JZQElAREdlXRRPAnYDnQGJhrZu2AjJ7Zf7/9whiDRo1Cj5GVK8t/jaTev/8dLvaDBsE334S+///8p0aOi5SlQonA3e9091buPtiDFUDGV5i0bQvTp4fVpAYPDj8lPX3+eejtk5MT1p343e9g0aLQG0yNvyJlq1AiMLNmZnZ78ZoAZvb/CKWDjJeTs3NE6dCh4VumpI+NG8N04ocfHtp2rroKli6Fq69W10+Riqpo1dADwHrgnOi2DngwrqDSzXe/G+aXeemlMLWAVjhLve3bw4W/U6fQGHzyySFZ33477L9/qqMTqV0q2mvoMHc/M+Hx/2XKFBMVdd55oRvi5ZeH+3l5Gn2cKrNnw09+EgaE9eoVGoH79Ut1VCK1V0VLBJvM7NvFD8ysL7ApnpDST15eGHF6xRWw774wdWpY7nL79lRHll3efx+GDIETToDVq8Pf5fXXlQREqqqiJYKxwCNmVrzM9n+BUfGElF7y8sIqZhs3hsdr1oT1DB56CBo2hMmT1RgZp02bwkjgxx4LDfeNGsGvfx3aAho2THV0IpmholNMvAN0M7N9osfrzOxqYGGMsaWF8eN3JoFiW7eG2SfvvjtcjCZNUjKoTtu3hwXfH300TAC4bl1YSe4nPwk3jQQWqV6VWkfJ3RPHDvwYuKNao0lDn3ySfPu6dWFuottvD1MW3HxzzcaViRYvDhf/vLwwzUeTJmFG2B/8IIwGVpuMSDyqsqBeVnwHbtsWVqzYfXu7dnDHHaHq4pZbQsng+utrPLxa77PPwmpfjz4aGn/r1Ak9gH7729AeoLUhROJXlUSQsVNMJJo4cdc2AggXp4kTw7xE99wTksHPfx62X3VV6mKtLf73vzCFx2OPwQsvhKqg3NwwKdywYWHlOBGpOWUmAjNbT/ILvgHlNtWZ2UDg94RFbO5z91tLPD8WuJwwq+kGYIy7L65Y6DVjZLQY5/jxoZqobduQBIq316kTGo43bQqDmBo2DIlDdtq+PawP/c47odrn738PyaBdu1CK+sEPoHPnVEcpkr0qtHj9Hh3YrA7wAfBdoJAwffWIxAu9me1T3O5gZkOAH7r7wLKOW52L11enLVvCBHUzZsAjj4SLWzbYti1M71BYuPP26ae7Pl65MswECtCsWRiUd955Yb3fvSragVlEqqQ6Fq/fE72Bpe6+LAoiHzgN2JEISjQ+N6YWVzftvXfo4XLqqWH66gYNQkNnpnjvPXj++VAqSrzQ/+c/IRkkql8/rOXQpk3o49+6dbgdemiY0rtBg9T8DiKSXJyJoBXwacLjQuCYkjuZ2eWEHkh7AyckO5CZjQHGALRt27baA60uDRvCP/4R1rkdMSI8PuWUVEe155YvDw25+fmhWgfC71R8kT/++J0X+TZtdt5v3lzdaUVqkzgTQYW4+2RgspmdC/yCJAPV3H0KMAVC1VDNRlg5TZrAs8/CSSfBmWeGJRFPOinVUVXcZ5/B44+Hi//rr4dtffqEhtyhQ6FVK13kRTJNnIlgJdAm4XHraFtp8oG7Y4ynxjRrBjNnhr7vp50WFkmpyjQIW7eGb+dffQUdOoReNdV5MV69Gv72tzBnz5w5YWH3bt3C8o7DhoXpNUQkc8WZCOYBHc2sAyEBDAfOTdzBzDq6+4fRw1OAD8kQ++8fukZ+5zuheujFF6F379L337YtjFf48MPdb8uX72xsBWjaNMy62alTWEWt+H6nTiEJVcT69aEaKz8/JK2ionCsG24ISzp+61tV+vVFpBaJLRG4e5GZXQHMJHQffcDdF5nZzUCBu08DrjCzk4CtZOD8RQceCLNmQf/+YZDUrFmh/jzZxX7Zsl2nt27cOFyYu3cPvWw6doSWLeHjj+GDD8Lt9dfDhTyx49cBB+yaGIoTxeGHh/1mzAjf/J95BjZvDnX711wT2jRyclTtI5KNYus+Gpd07T5aluXLQ9VQYeGu2xs2DBfojh13vx10UMUuyt98E5JIcXIovn34YajvL2YWevNs3hySxTnnhG/+ffqoC6dINkhV91GJtG8fJlF75BE4+OCdF/tDDqn6Rbh+/VCNk6wqZ/36kBCKk8N//xuqqQYMgLr6y4tIRCUCEZEsUFaJQJUCIiJZTolARCTLKRGIiGQ5JQIRkSynRCAikuWUCEREspwSgYhIllMiEBHJckoEIiJZTomgBuTlhWkm9tor/MzLS3VEIiI7acaZmOXlhcXsN24Mj1es2Lm4/ciRqYtLRKSYSgQxGz9+ZxIotnFj2C4ikg6UCGL2ySeV2y4iUtOUCGLWtm3ltouI1DQlgphNnAiNGu26rVGjsF1EJB0oEcRs5EiYMgXatQurhLVrFx6roVhE0oV6DdWAkSN14ReR9BVricDMBprZ+2a21MzGJXn+x2a22MwWmtksM2sXZzwiIrK72BKBmdUBJgODgC7ACDPrUmK3t4Bcd+8KPAn8Nq54REQkuThLBL2Bpe6+zN23APnAaYk7uPtsdy/uZf860DrGeEREJIk4E0Er4NOEx4XRttJcBMyIMR4REUkiLRqLzewHQC7wnVKeHwOMAWirDvgiItUqzhLBSqBNwuPW0bZdmNlJwHhgiLt/k+xA7j7F3XPdPbdly5axBCsikq3iTATzgI5m1sHM9gaGA9MSdzCz7sCfCEngyxhjqdU0e6mIxCm2qiF3LzKzK4CZQB3gAXdfZGY3AwXuPg24DWgCPGFmAJ+4+5C4YqqNNHupiMTN3D3VMVRKbm6uFxQUpDqMGtO+fbj4l9SuHSxfXtPRiEhtZWbz3T032XOaYiLNafZSEYmbEkGa0+ylIhI3JYI0p9lLRSRuSgRpTrOXikjc0mJAmZRNs5eKSJxUIhARyXJKBFlAA9JEpCyqGspwGpAmIuVRiSDDjR+/MwkU27gxbBcRASWCjKcBaSJSHiWCDKcBaSJSHiWCDKcBaSJSHiWCDKcBaSJSHiWCLDByZJipdPv28LOySUDdT0Uym7qPSpnU/VQk86lEIGVS91ORzKdEIGVS91ORzKdEIGVS91ORzKdEIGVS91ORzKdEIGWqju6n6nUkkt5iTQRmNtDM3jezpWY2Lsnz/c1sgZkVmdlZccYie64q3U+Lex2tWAHuO3sdKRmIpI/YEoGZ1QEmA4OALsAIM+tSYrdPgNHAn+OKQ1JLvY5E0l+c4wh6A0vdfRmAmeUDpwGLi3dw9+XRc9tjjENSSL2ORNJfnFVDrYBPEx4XRtsqzczGmFmBmRWsWrWqWoKTmqFeRyLpr1Y0Frv7FHfPdffcli1bpjocqYTq6HWkxmaReMWZCFYCbRIet462SRapaq8jNTaLxM/cPZ4Dm9UFPgBOJCSAecC57r4oyb4PAc+4+5PlHTc3N9cLCgqqOVpJV+3bh4t/Se3ahR5MIlIxZjbf3XOTPRdbicDdi4ArgJnAe8Dj7r7IzG42syFRYL3MrBA4G/iTme2WJCS7qbFZJH6xzj7q7tOB6SW23Zhwfx6hykgkqbZtk5cI1NgsUn1qRWOxZC81NovET4lA0poam0Xip0Qgaa8qU1xUx8hmlSgk02mFMsloVW1s1gptkg1UIpCMVtWRzZorSbKBEoFktKo2Nqv7qmQDJQLJaFVtbK6OuZLUxiDpTolAMl5VGpurWqJQryWpDZQIRMpQ1RJFdbUxqFQhcYptrqG4aK4hqU322iuUBEoyCyWUiijZcwlCqaSyS4ZKdkvJXEMiUj1tDBoLIXFTIhCJUXVMkVFdYyGq0k6hRJLZlAhEYlTVNgZI/VgINXhnPrURiKS5qrYRVLWdQmtCZAa1EYjUYqkeC1Edg+pUtZTelAhEaoFUjoWoaiJJhzYKJaJyuHutuvXs2dNFpHIee8y9XTt3s/Dzsccq99pGjdzDZTzcGjWq+DHatdv1tcW3du1q5v2r+vriY+zp+UsXQIGXcl1N+YW9sjclApGaV5ULoVnyRGBWsddXNZGkOhEVHyPViaSsRKDGYhGJVVUbm6va2J3qxvLqGBCYlxd6eX3ySaiSmzix8oMJU9ZYbGYDzex9M1tqZuOSPF/fzKZGz79hZu3jjEdEal6q2yhS3VheG7rvxpYIzKwOMBkYBHQBRphZlxK7XQT8190PB34H/CaueEQkNara66mqiSTViSjViaRCSqszquoN6APMTHh8PXB9iX1mAn2i+3WBr4jGNpR2UxuBSPapah17bW4sr2obSzHKaCOIs2qoFfBpwuPCaFvSfdy9CFgLNC95IDMbY2YFZlawatWqmMIVkXRVle6zVX19qks01TFfVXlqxTgCd5/i7rnuntuyZctUhyMiWaY2J5KKiHPx+pVAm4THraNtyfYpNLO6QDNgdYwxiYjUuJEj93zK8OLXVbXXUFniTATzgI5m1oFwwR8OnFtin2nAKOBfwFnAS1FdloiIRKqSSCoitkTg7kVmdgWhQbgO8IC7LzKzmwmNFtOA+4FHzWwp8DUhWYiISA2Ks0SAu08HppfYdmPC/c3A2XHGICIiZasVjcUiIhIfJQIRkSynRCAikuVq3aRzZrYKSDIFVFpoQRgdna4UX9Wke3yQ/jEqvqqpSnzt3D3pQKxalwjSmZkVeCmz+6UDxVc16R4fpH+Miq9q4opPVUMiIllOiUBEJMspEVSvKakOoByKr2rSPT5I/xgVX9XEEp/aCEREspxKBCIiWU6JQEQkyykRVJKZtTGz2Wa22MwWmdlVSfYZYGZrzezt6HZjsmPFGONyM/t39N4FSZ43M7szWit6oZn1qMHYjkg4L2+b2Tozu7rEPjV+/szsATP70szeTdi2v5m9YGYfRj/3K+W1o6J9PjSzUTUU221mtiT6+/3dzPYt5bVlfhZijnGCma1M+DsOLuW1Za5tHmN8UxNiW25mb5fy2ljPYWnXlBr9/JW2dJlupS7BeTDQI7rfFPgA6FJinwHAMymMcTnQooznBwMzAAOOBd5IUZx1gM8JA11Sev6A/kAP4N2Ebb8FxkX3xwG/SfK6/YFl0c/9ovv71UBs3wPqRvd/kyy2inwWYo5xAvDTCnwGPgIOBfYG3in5/xRXfCWe/3/Ajak4h6VdU2ry86cSQSW5+2fuviC6vx54j92X4Ex3pwGPePA6sK+ZHZyCOE4EPnL3lI8Ud/e5hKnQE50GPBzdfxg4PclLTwZecPev3f2/wAvAwLhjc/fnPSzvCvA6YeGnlCnl/FVEb2Cpuy9z9y1APuG8V6uy4jMzA84B/lLd71sRZVxTauzzp0RQBWbWHugOvJHk6T5m9o6ZzTCzI2s2Mhx43szmm9mYJM9XZD3pmjCc0v/5Unn+ih3o7p9F9z8HDkyyTzqcywsJJbxkyvssxO2KqPrqgVKqNtLh/PUDvnD3D0t5vsbOYYlrSo19/pQI9pCZNQH+Clzt7utKPL2AUN3RDfgD8FQNh/dtd+8BDAIuN7P+Nfz+5TKzvYEhwBNJnk71+duNh3J42vW1NrPxQBGQV8ouqfws3A0cBuQAnxGqX9LRCMouDdTIOSzrmhL350+JYA+YWT3CHyzP3f9W8nl3X+fuG6L704F6ZtaipuJz95XRzy+BvxOK34kqsp503AYBC9z9i5JPpPr8JfiiuMos+vllkn1Sdi7NbDRwKjAyulDspgKfhdi4+xfuvs3dtwP3lvLeKf0sWlgrfSgwtbR9auIclnJNqbHPnxJBJUX1ifcD77n77aXsc1C0H2bWm3CeV9dQfI3NrGnxfUKj4rsldpsGnG/BscDahCJoTSn1W1gqz18JxWtqE/38R5J9ZgLfM7P9oqqP70XbYmVmA4HrgCHuvrGUfSryWYgzxsR2pzNKee8da5tHpcThhPNeU04Clrh7YbIna+IclnFNqbnPX1wt4Zl6A75NKKItBN6OboOBscDYaJ8rgEWEHhCvA8fVYHyHRu/7ThTD+Gh7YnwGTCb01vg3kFvD57Ax4cLeLGFbSs8fISl9Bmwl1LNeBDQHZgEfAi8C+0f75gL3Jbz2QmBpdLughmJbSqgbLv4M3hPtewgwvazPQg2ev0ejz9dCwkXt4JIxRo8HE3rKfBRXjMnii7Y/VPy5S9i3Rs9hGdeUGvv8aYoJEZEsp6ohEZEsp0QgIpLllAhERLKcEoGISJZTIhARyXJKBCIRM9tmu86MWm0zYZpZ+8SZL0XSSd1UByCSRja5e06qgxCpaSoRiJQjmo/+t9Gc9G+a2eHR9vZm9lI0qdosM2sbbT/QwhoB70S346JD1TGze6M55583s4bR/ldGc9EvNLP8FP2aksWUCER2aliiamhYwnNr3f1o4I/AHdG2PwAPu3tXwqRvd0bb7wT+6WHSvB6EEakAHYHJ7n4ksAY4M9o+DugeHWdsPL+aSOk0slgkYmYb3L1Jku3LgRPcfVk0Odjn7t7czL4iTJuwNdr+mbu3MLNVQGt3/ybhGO0J88Z3jB7/DKjn7r80s+eADYRZVp/yaMI9kZqiEoFIxXgp9yvjm4T729jZRncKYe6nHsC8aEZMkRqjRCBSMcMSfv4ruv8aYbZMgJHAy9H9WcBlAGZWx8yalXZQM9sLaOPus4GfAc2A3UolInHSNw+RnRrarguYP+fuxV1I9zOzhYRv9SOibT8CHjSza4FVwAXR9quAKWZ2EeGb/2WEmS+TqQM8FiULA+509zXV9PuIVIjaCETKEbUR5Lr7V6mORSQOqhoSEclyKhGIiGQ5lQhERLKcEoGISJZTIhARyXJKBCIiWU6JQEQky/1/x4sIaCwEd/4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 766 ms (started: 2021-07-22 17:00:12 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "history_dict = history.history\n",
    "loss_values = history_dict[\"loss\"]\n",
    "val_loss_values = history_dict[\"val_loss\"]\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "plt.plot(epochs, loss_values, \"bo\", label=\"Training loss\")\n",
    "plt.plot(epochs, val_loss_values, \"b\", label=\"Validation loss\")\n",
    "plt.title(\"Training and validation loss\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "satellite-crash",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsmklEQVR4nO3deXxV1bn/8c+TMBlAZhSZghZx+ClTChUnnHG4Ury2ilSh2oug1uqv1NpqLdX6a23trS+rtY11qtKLem0t3osjTm21loCAIxoxahA1okwiMuT5/bH2CYewT3KSMyXk+3699uvs+Txn52Q/Z62199rm7oiIiNRXVOgARESkZVKCEBGRWEoQIiISSwlCRERiKUGIiEgsJQgREYmlBCFpM7OHzWxqttctJDOrMrNjc7BfN7MvReO/M7MfpbNuM95nipk91tw4RRpiug9i12ZmG5ImS4AvgG3R9PnuPif/UbUcZlYFfMvdn8jyfh0Y6u6V2VrXzEqBt4H27r41K4GKNKBdoQOQ3HL3Lonxhk6GZtZOJx1pKfR9bBlUxdRGmdl4M6s2s++b2QfAHWbWw8z+x8xqzOzTaHxA0jZPm9m3ovFpZvZ3M7s+WvdtMzuxmesOMbNnzWy9mT1hZjeb2T0p4k4nxmvM7B/R/h4zs95Jy882s3fMbLWZXdHA8RlrZh+YWXHSvElmtiwaH2Nmz5vZGjNbZWY3mVmHFPu608x+mjT9vWib983s3HrrnmxmL5rZOjN7z8xmJy1+NnpdY2YbzOyQxLFN2n6cmS00s7XR67h0j00Tj3NPM7sj+gyfmtmDScsmmtmS6DO8ZWYTovk7VOeZ2ezE39nMSqOqtvPM7F3gyWj+/dHfYW30HTkwafvdzOxX0d9zbfQd283M/tfMvl3v8ywzs0lxn1VSU4Jo2/YEegKDgemE78Md0fQg4HPgpga2HwssB3oDvwBuMzNrxrp/Av4F9AJmA2c38J7pxHgW8E2gL9ABmAVgZgcAt0T73yt6vwHEcPcXgM+Ao+vt90/R+Dbg0ujzHAIcA1zQQNxEMUyI4jkOGArUb//4DDgH6A6cDMw0s69Gy46IXru7exd3f77evnsC/wvcGH22/wT+18x61fsMOx2bGI0d57sJVZYHRvv6dRTDGOCPwPeiz3AEUJXiPeIcCewPnBBNP0w4Tn2BxUBylej1wGhgHOF7fBlQC9wFfCOxkpkNB/oTjo00hbtraCMD4R/12Gh8PLAZ6NTA+iOAT5OmnyZUUQFMAyqTlpUADuzZlHUJJ5+tQEnS8nuAe9L8THExXpk0fQHwSDR+FTA3aVnn6Bgcm2LfPwVuj8a7Ek7eg1Osewnwl6RpB74Ujd8J/DQavx34edJ6+yavG7PfG4BfR+Ol0brtkpZPA/4ejZ8N/Kve9s8D0xo7Nk05zkA/wom4R8x6v0/E29D3L5qenfg7J322vRuIoXu0TjdCAvscGB6zXifgU0K7DoRE8ttc/E/t6oNKEG1bjbtvSkyYWYmZ/T4qsq8jVGl0T65mqeeDxIi7b4xGuzRx3b2AT5LmAbyXKuA0Y/wgaXxjUkx7Je/b3T8DVqd6L0Jp4TQz6wicBix293eiOPaNql0+iOL4f4TSRGN2iAF4p97nG2tmT0VVO2uBGWnuN7Hvd+rNe4fw6zkh1bHZQSPHeSDhb/ZpzKYDgbfSjDdO3bExs2Iz+3lUTbWO7SWR3tHQKe69ou/0vcA3zKwImEwo8UgTKUG0bfUvYfsuMAwY6+67s71KI1W1UTasAnqaWUnSvIENrJ9JjKuS9x29Z69UK7v7q4QT7InsWL0EoarqdcKv1N2BHzYnBkIJKtmfgHnAQHfvBvwuab+NXXL4PqFKKNkgYGUacdXX0HF+j/A36x6z3XvAPin2+Rmh9JiwZ8w6yZ/xLGAioRquG6GUkYjhY2BTA+91FzCFUPW30etVx0l6lCAkWVdCsX1NVJ/941y/YfSLvAKYbWYdzOwQ4N9yFON/A6eY2WFRg/LVNP4/8CfgO4QT5P314lgHbDCz/YCZacZwHzDNzA6IElT9+LsSfp1viurzz0paVkOo2tk7xb7nA/ua2Vlm1s7MzgAOAP4nzdjqxxF7nN19FaFt4LdRY3Z7M0skkNuAb5rZMWZWZGb9o+MDsAQ4M1q/DDg9jRi+IJTySgiltEQMtYTquv80s72i0sYhUWmPKCHUAr9CpYdmU4KQZDcAuxF+nf0TeCRP7zuF0NC7mlDvfy/hxBDnBpoZo7u/AlxIOOmvItRTVzey2X8RGk6fdPePk+bPIpy81wO3RjGnE8PD0Wd4EqiMXpNdAFxtZusJbSb3JW27EbgW+IeFq6e+Um/fq4FTCL/+VxMabU+pF3e6bqDh43w2sIVQivqI0AaDu/+L0Aj+a2At8AzbSzU/Ivzi/xT4CTuWyOL8kVCCWwm8GsWRbBbwErAQ+AS4jh3PaX8EDiK0aUkz6EY5aXHM7F7gdXfPeQlGdl1mdg4w3d0PK3QsrZVKEFJwZvZlM9snqpKYQKh3frDAYUkrFlXfXQCUFzqW1kwJQlqCPQmXYG4gXMM/091fLGhE0mqZ2QmE9poPabwaSxqgKiYREYmlEoSIiMTaZTrr6927t5eWlhY6DBGRVmXRokUfu3ufuGW7TIIoLS2loqKi0GGIiLQqZlb/7vs6qmISEZFYShAiIhJLCUJERGIpQYiISCwlCBERiZWzBGFmt5vZR2b2corlZmY3mlll9DjAUUnLpprZm9EwNVcxioi0ZnPmQGkpFBWF1zlzGtuiaXJZgrgTmNDA8hMJjxIcSnjc5S1Q99jEHxMeUTkG+LGZ9chhnCIizZLpCTqT7efMgenT4Z13wD28Tp+e3SSRswTh7s8SuuBNZSLwRw/+SXhaVT/Cs2gfd/fEE6sep+FEIyJtVK5/QTf23pmcoDPd/oorYOPGHedt3BjmZ0sh2yD6s+OjF6ujeanm78TMpptZhZlV1NTU5CxQEcmNQv+CzuT9Mz1BZ7r9u+82bX5ztOpGancvd/cydy/r0yf2TnERaaEK/Qs60/fP9ASd6faD6j+stpH5zVHIBLGSHZ/NOyCal2q+iOxCCv0LOtP3z/QEnen2114LJSU7zispCfOzpZAJYh5wTnQ101eAtdGzbh8Fjo+eddsDOD6aJyJZVshG1kL/gs70/TM9QWe6/ZQpUF4OgweDWXgtLw/zs8bdczIQnuW7ivDc2mrgPGAGMCNabsDNwFuE58qWJW17LuF5vZXAN9N5v9GjR7uIpO+ee9xLStxDBUsYSkrC/HxsP3jwjtsmhsGDW8f7J2IYPNjdLLym+97Z2j4bgApPdR5PtaC1DUoQIk2T6Qmy0Cf4xD6ae4LNxvvvChpKEK26kVqkrStkFU+m22ejimTKFKiqgtra8NrUbXNeRdPK7TKPHC0rK3M9D0LaksRVOMkNrSUl6Z/kSkvDlTv1DR4cTra53l5aBjNb5O5lcctUghBppTK9CqfQjazS8ilBiBRQIauIMq1iURXNrk9VTCIFUugqIhFQFZNIi1ToKiKRxihBiGSgNVcRiTSmXaEDEGmt6lcRJfrygfRO0oMGxVcRNaUvnSlTlBAkd1SCEGkmVRHJrk4JQto0VRGJpKYqJmmzVEUk0jCVIKTNUhWRSMOUIKTNUhWRSMNUxSRtlqqIRBqmEoS0WaoiEmmYEoS0aplchaQqIpGGqYpJWq1Mr0JKrKeEIBJPJQhptTK9CklEGqYEIa1WplchiUjDlCCkoDJpQ0h1tVFTrkISkdSUIKRgEm0I77wTHhmfaENIN0noKiSR3FKCkILJtA1BVyGJ5JaeKCcFU1QUSg71mUFtbf7jEWmL9EQ5aZHUhiDSsilBSMGoDUGkZVOCkIJRG4JIy6Y7qaWgdCezSMulEoRkJJP7GESkZVMJQpotG30hiUjLpRKENJv6QhLZtSlBSLOpLySRXZsShDSb7mMQ2bUpQUiz6T4GkV2bEoQ0m+5jENm16SomyYjuYxDZdakEISIisZQgREQkVk4ThJlNMLPlZlZpZpfHLB9sZgvMbJmZPW1mA5KWbTOzJdEwL5dxtmW6E1pEUslZG4SZFQM3A8cB1cBCM5vn7q8mrXY98Ed3v8vMjgZ+BpwdLfvc3UfkKj7RndAi0rBcliDGAJXuvsLdNwNzgYn11jkAeDIafypmueSQ7oQWkYbkMkH0B95Lmq6O5iVbCpwWjU8CuppZr2i6k5lVmNk/zeyrcW9gZtOjdSpqamqyGHrboDuhRaQhhW6kngUcaWYvAkcCK4Ft0bLB0WPwzgJuMLN96m/s7uXuXubuZX369Mlb0LsK3QktIg3JZYJYCQxMmh4Qzavj7u+7+2nuPhK4Ipq3JnpdGb2uAJ4GRuYw1jZJd0KLSENymSAWAkPNbIiZdQDOBHa4GsnMeptZIoYfALdH83uYWcfEOsChQHLjtmSB7oQWkYbk7Comd99qZhcBjwLFwO3u/oqZXQ1UuPs8YDzwMzNz4Fngwmjz/YHfm1ktIYn9vN7VT5IluhNaRFIxdy90DFlRVlbmFRUVhQ5DRKRVMbNFUXvvTgrdSC0Z0o1uIpIr6qyvFdONbiKSSypBtGK60U1EckkJohXTjW4ikktKEK2YbnQTkVxSgmjFdKObiOSSEkQrphvdRCSXdBVTK6cb3UQkV1SCEBGRWEoQIiISSwlCRERiKUGIiEgsJQgREYmlBFFg6mxPRFoqXeZaQOpsT0RaMpUgCkid7YlIS6YEUUDqbE9EWjIliAJSZ3si0pIpQRSQOtsTkZZMCaKA1NmeiLRkuoqpwNTZnoi0VCpBiIhILCUIERGJpQQhIiKxlCBERCSWEoSIiMRSghARkViNJggz+zczUyIREWlj0jnxnwG8aWa/MLP9ch2QiIi0DI0mCHf/BjASeAu408yeN7PpZtY159GJiEjBpFV15O7rgP8G5gL9gEnAYjP7dg5jExGRAkqnDeJUM/sL8DTQHhjj7icCw4Hv5jY8EREplHT6Yvp34Nfu/mzyTHffaGbn5SYsEREptHQSxGxgVWLCzHYD9nD3KndfkKvARESksNJpg7gfqE2a3hbNExGRXVg6CaKdu29OTETjHXIXkoiItATpJIgaMzs1MWFmE4GPcxdS6zJnDpSWQlFReJ0zp9ARiYhkRzptEDOAOWZ2E2DAe8A5OY2qlZgzB6ZPh40bw/Q774Rp0EOARKT1S+dGubfc/SvAAcD+7j7O3SvT2bmZTTCz5WZWaWaXxywfbGYLzGyZmT1tZgOSlk01szejYWpTPlS+XHHF9uSQsHFjmC8i0tql9chRMzsZOBDoZGYAuPvVjWxTDNwMHAdUAwvNbJ67v5q02vXAH939LjM7GvgZcLaZ9QR+DJQBDiyKtv20SZ8ux959t2nzRURak3RulPsdoT+mbxOqmL4GDE5j32OASndfETVszwUm1lvnAODJaPyppOUnAI+7+ydRUngcmJDGe+bVoEFNmy8i0pqk00g9zt3PAT51958AhwD7prFdf0J7RUJ1NC/ZUuC0aHwS0NXMeqW5LVGfUBVmVlFTU5NGSNl17bVQUrLjvJKSMF9EpLVLJ0Fsil43mtlewBZCf0zZMAs40sxeBI4EVhLus0iLu5e7e5m7l/Xp0ydLIaVvyhQoL4fBg8EsvJaXq4FaRHYN6bRBPGRm3YFfAosJbQK3prHdSmBg0vSAaF4dd3+fqARhZl2Af3f3NWa2Ehhfb9un03jPvJsyRQlBRHZNDZYgogcFLXD3Ne7+AKHtYT93vyqNfS8EhprZEDPrAJwJzKu3/95JDyP6AXB7NP4ocLyZ9TCzHsDx0TwREcmTBhOEu9cSrkRKTH/h7mvT2bG7bwUuIpzYXwPuc/dXzOzqpBvvxgPLzewNYA/g2mjbT4BrCElmIXB1NE9ERPLE3L3hFcyuB54H/uyNrVxAZWVlXlFRUegwRERaFTNb5O5lccvSaaQ+n9A53xdmts7M1pvZuqxG2Ia5w29+AwsWhHERkZYinTupu7p7kbt3cPfdo+nd8xFcW1BeDhdfDMceC2PHwoMPQm1to5uJiORcOjfKHRE35CO4XV1VFcyaBcccA7//PaxeDZMmwUEHwd13w5YthY5QRNqydKqYvpc0/Ah4iPAQIcmAO3zrW+H+idtuC538LV8Of/oTFBfDOefA0KFw883w+eeFjlZE2qJ0qpj+LWk4Dvg/QIvqE6k1+v3vQ7vD9deHG+wA2rWDyZNh6VJ46CHYay+46KLQjfjPfw5r07p+rOnWrIFtad+eKCJtRaNXMe20Qeit7xV3PyA3ITVPa7qKqaoqVCMdcgg8+mgoRcRxh2efhZ/9LKzXrRtceCF85zvQt2/z3nvVKli8GBYtCq+LF8N770GvXnDccXDCCXD88SE5iciur6GrmNK5zPU3hLunIZQ4RgBV7v6NbAaZqdaSIGprw4l44UJ4+eX0O/ZbtCiUIh54ADp1CtVTs2al3t4dqqt3TgaroqeLm8G++8KoUSFZvf46PPYYfPBBWH7QQSFZnHACHHZYeE8R2fVkmiCSn8WwlZAc/pHF+LKitSSIW26BCy4IVUyJhws1xeuvwy9+ERqxAb7xDfj+96Fjx52TQaL/wqIi2H//kAxGjw6vI0ZA16477tsdli0LpZVHH4W//x02b4bddoPx47cnjGHDUpd6RKR1yTRBdAY2ufu2aLoY6OjuGxvcMM9aQ4J4++3wy3zcuIarltLx7rvwq1/Brbfu2Ijdrh0ceOD2RDBqFAwfvnOvs+n47DN4+untCeONN8L8QYO2J4tjjoHu3Zv/OUSksDJNEP8EjnX3DdF0F+Axdx+X9Ugz0NITRG1tuNehoqJpVUuNqamBu+4KpYFEdVGuqoPefjtUQz36aGhgX7cuXHE1diz06xdKILW14TV5PNVr8ni3brDHHrDnntuH5OkuXXLzmUTaukwTxBJ3H9HYvEJr6QkiUbVUXg7/8R+FjiZzW7bACy/AI4/Ak0+GK6yKikKpKPGaPJ7qNTGsWRPaPz76KP6O8s6dd0wY9ZPJsGHhsuCidC7cFpE6DSWIdLr7/szMRrn74mhnowFdmd8Eb78N3/teuDroW98qdDTZ0b59aLw+7LDs7nfbNvj445AsPvgAPvxw5/HXX4dnngk3Fibr0gVGjtyxrWXYsFDtJiJNl04J4suEx4W+T3jk6J7AGe6+KPfhpa+lliByVbUkoQG9pgbefx9eeWV7A/2SJbAxaiHbbbfQIJ9ojxk1KrTRtG9fyMhFWo6MqpiiHbQHhkWTy929xXUC0VITxG9/G+5duPXWXaf00NJt2xbuSk++quvFF2H9+rC8Qwc4+ODtJY2RI6Fnz1BttnVr469x87Zty6wNplcv2HvvMJSWhqvSRPIh0zaIC4E57r4mmu4BTHb332Y70Ey0xASRuGrp0ENDXb0uDS2c2lqorNx+CXAicaxZU+jIdmYG/fuHZLHPPtsTR2Lo00ffJcmeXDRSv+juI7MXYuZaWoKorQ2XgC5apKqllso93NW+eHG4pLd9+9Be0dTXdu3C1VxNbZxPHv/oI1ixIn54//0d4+7ceeek0bcv7L57uJqta9cdxzt0KMjhlVYi00bqYjOzxMOCovsg9JVrxC23hHsIbr1VyaGlMoMhQ8JQaP36heHQQ3detnFjSGT1E0dlZbjsuLHOHDt2jE8cyePdukGPHuGelu7ddx7v3FmllrYonQTxCHCvmf0+mj4feDh3IbV+K1aEu5tPOAHOO6/Q0UhrV1ICBxwQhvrcw9VdH38c2ljWrQuviSF5Onm8pgbeemv7/M8+aziG4uLUyaN799Dh5MEHhypV3Ti560gnQXwfmA7MiKaXEa5kkhi1tSEpFBeH0oN+dUkumW2/FyQTW7eGe1nWrAnDp582Pl5dvX3eF19s39egQSFZJBLGwQeHfr90uXHr0+ifzN1rzewFYB/g60Bv4IFcB9ZaJaqW/vAHGDiw0NGIpKddu3AlVa9eTd/WHVauhJdeCn15JYZHHgmJB0I11wEHbE8ciaGxXondYdOmxktEJSXh6q8hQ8KrOpfMjpSN1Ga2LzA5Gj4G7gVmufvg/IWXvuY2Um/eDEcfHbrePuYYOPzwUN/aHCtWhF9MRxwB8+er9CBt2xdfhJsaly3bMXkkehSGkCAOPji0gaRKAs15Vsmee25vX0pOHEOGhBKO7oPZrllXMZlZLfA34Dx3r4zmrXD3vXMWaQaamyBWroSzzoLnnw/XtLdvH/oWOuaYMIwdm95VILW1IdG8+GK4akmlB5F4NTU7lzY2bkzdgJ7O+IYN4bLyt98ODfrJr+++u2OSKSoKlxEnJ42BA3fswqVv37Zz9VdzE8RXgTOBQwkN1XOBP7h7C7jmY2eZXua6cWPo3nrBgtC30KJFoXhbUhJKFYmEMXx4aF+o76ab4NvfDlVLapgWaTm2bg0/BOOSx9tvh2Vxp8GePXfu9ytuvE+f+HNCa5GN7r4nEqqajgb+CPzF3R/LdqCZyPZ9EJ9+GtoSnnwyJI3XXgvze/SAo44KyeLoo0NfPytWhGKyqpZEWp/Nm7f395Wq/6/EEHe1V1FRqJZu6D6Z5ixryn04/frBxInN+/wZd7WRtKMewNcIfTEd07xwciPXN8q9/z489VRIFgsWhGIrhEdzduoULjN85RUYMCBnIYhIgW3YsD1pJF5XrQrzG+qKJdNlW7aEIZWvfCVUkzdH1hJES5bPO6ndQ6khUR31/PPhudFnnZWXtxeRNmrbtvgEUlQUqryaI9M7qaUes9BHzj77NO+xoSIizVFcHIZ8deaox6uIiEgsJQgREYmlBCEiIrGUIEREJJYShIiIxFKCEBGRWEoQIiISSwlCRERiKUGIiEgsJQgREYmV0wRhZhPMbLmZVZrZ5THLB5nZU2b2opktM7OTovmlZva5mS2Jht/lMk4REdlZzvpiMrNi4GbgOKAaWGhm89z91aTVrgTuc/dbzOwAYD5QGi17y91H5Co+ERFpWC5LEGOASndf4e6bCQ8cqt9juQO7R+PdgPdzGI+IiDRBLhNEf+C9pOnqaF6y2cA3zKyaUHr4dtKyIVHV0zNmdnjcG5jZdDOrMLOKmpqaLIYuIiKFbqSeDNzp7gOAk4C7zawIWAUMcveRwP8F/mRmu9ff2N3L3b3M3cv69OmT18BFRHZ1uUwQK4GBSdMDonnJzgPuA3D354FOQG93/8LdV0fzFwFvAfvmMFYREaknlwliITDUzIaYWQfgTGBevXXeBY4BMLP9CQmixsz6RI3cmNnewFBgRQ5jFRGRenJ2FZO7bzWzi4BHgWLgdnd/xcyuBircfR7wXeBWM7uU0GA9zd3dzI4ArjazLUAtMMPdP8lVrCIisjM9k1pEpA1r6JnUhW6kFhGRFkoJQkREYilBiIhILCUIERGJpQQhIiKxlCBERCSWEoSIiMRSghARkVhKECIiEksJQkREYilBiIhILCUIERGJpQQhIiKxlCBERCSWEoSIiMRSghARkVhKECIiEksJQkREYilBiIhILCUIERGJpQQhIiKxlCBERCSWEoSIiMRSghARkVhKECIiEqtdoQMQkdZvy5YtVFdXs2nTpkKHIil06tSJAQMG0L59+7S3UYIQkYxVV1fTtWtXSktLMbNChyP1uDurV6+murqaIUOGpL2dqphEJGObNm2iV69eSg4tlJnRq1evJpfwlCBEJCuUHFq25vx9lCBERCSWEoSI5N2cOVBaCkVF4XXOnMz2t3r1akaMGMGIESPYc8896d+/f9305s2bG9y2oqKCiy++uNH3GDduXGZBtkJqpBaRvJozB6ZPh40bw/Q774RpgClTmrfPXr16sWTJEgBmz55Nly5dmDVrVt3yrVu30q5d/OmurKyMsrKyRt/jueeea15wrZhKECKSV1dcsT05JGzcGOZn07Rp05gxYwZjx47lsssu41//+heHHHIII0eOZNy4cSxfvhyAp59+mlNOOQUIyeXcc89l/Pjx7L333tx44411++vSpUvd+uPHj+f0009nv/32Y8qUKbg7APPnz2e//fZj9OjRXHzxxXX7TVZVVcXhhx/OqFGjGDVq1A6J57rrruOggw5i+PDhXH755QBUVlZy7LHHMnz4cEaNGsVbb72V3QPVAJUgRCSv3n23afMzUV1dzXPPPUdxcTHr1q3jb3/7G+3ateOJJ57ghz/8IQ888MBO27z++us89dRTrF+/nmHDhjFz5syd7h148cUXeeWVV9hrr7049NBD+cc//kFZWRnnn38+zz77LEOGDGHy5MmxMfXt25fHH3+cTp068eabbzJ58mQqKip4+OGH+etf/8oLL7xASUkJn3zyCQBTpkzh8ssvZ9KkSWzatIna2trsH6gUlCBEJK8GDQrVSnHzs+1rX/saxcXFAKxdu5apU6fy5ptvYmZs2bIldpuTTz6Zjh070rFjR/r27cuHH37IgAEDdlhnzJgxdfNGjBhBVVUVXbp0Ye+99667z2Dy5MmUl5fvtP8tW7Zw0UUXsWTJEoqLi3njjTcAeOKJJ/jmN79JSUkJAD179mT9+vWsXLmSSZMmAeFmt3xSFZOI5NW110J0DqxTUhLmZ1vnzp3rxn/0ox9x1FFH8fLLL/PQQw+lvCegY8eOdePFxcVs3bq1Weuk8utf/5o99tiDpUuXUlFR0WgjeiEpQYhIXk2ZAuXlMHgwmIXX8vLmN1Cna+3atfTv3x+AO++8M+v7HzZsGCtWrKCqqgqAe++9N2Uc/fr1o6ioiLvvvptt27YBcNxxx3HHHXewMWqg+eSTT+jatSsDBgzgwQcfBOCLL76oW54POU0QZjbBzJabWaWZXR6zfJCZPWVmL5rZMjM7KWnZD6LtlpvZCbmMU0Tya8oUqKqC2trwmuvkAHDZZZfxgx/8gJEjRzbpF3+6dtttN377298yYcIERo8eTdeuXenWrdtO611wwQXcddddDB8+nNdff72ulDNhwgROPfVUysrKGDFiBNdffz0Ad999NzfeeCMHH3ww48aN44MPPsh67KlYovU96zs2KwbeAI4DqoGFwGR3fzVpnXLgRXe/xcwOAOa7e2k0/l/AGGAv4AlgX3fflur9ysrKvKKiIiefRUQa9tprr7H//vsXOoyC27BhA126dMHdufDCCxk6dCiXXnppocOqE/d3MrNF7h57nW8uSxBjgEp3X+Hum4G5wMR66ziwezTeDXg/Gp8IzHX3L9z9baAy2p+ISIt16623MmLECA488EDWrl3L+eefX+iQMpLLq5j6A+8lTVcDY+utMxt4zMy+DXQGjk3a9p/1tu1f/w3MbDowHWBQLi6BEBFpgksvvbRFlRgyVehG6snAne4+ADgJuNvM0o7J3cvdvczdy/r06ZOzIEVE2qJcliBWAgOTpgdE85KdB0wAcPfnzawT0DvNbUVEJIdyWYJYCAw1syFm1gE4E5hXb513gWMAzGx/oBNQE613ppl1NLMhwFDgXzmMVURE6slZCcLdt5rZRcCjQDFwu7u/YmZXAxXuPg/4LnCrmV1KaLCe5uGyqlfM7D7gVWArcGFDVzCJiEj25bQNwt3nu/u+7r6Pu18bzbsqSg64+6vufqi7D3f3Ee7+WNK210bbDXP3h3MZp4i0bkcddRSPPvroDvNuuOEGZs6cmXKb8ePHk7g0/qSTTmLNmjU7rTN79uy6+xFSefDBB3n11bqr97nqqqt44oknmhB9y1XoRmoRkYxNnjyZuXPn7jBv7ty5KTvMq2/+/Pl07969We9dP0FcffXVHHvssQ1s0Xqosz4RyapLLoHo0QxZM2IE3HBD6uWnn346V155JZs3b6ZDhw5UVVXx/vvvc/jhhzNz5kwWLlzI559/zumnn85PfvKTnbYvLS2loqKC3r17c+2113LXXXfRt29fBg4cyOjRo4Fwj0N5eTmbN2/mS1/6EnfffTdLlixh3rx5PPPMM/z0pz/lgQce4JprruGUU07h9NNPZ8GCBcyaNYutW7fy5S9/mVtuuYWOHTtSWlrK1KlTeeihh9iyZQv3338/++233w4xVVVVcfbZZ/PZZ58BcNNNN9U9tOi6667jnnvuoaioiBNPPJGf//znVFZWMmPGDGpqaiguLub+++9nn332yei4qwQhIq1ez549GTNmDA8/HGqj586dy9e//nXMjGuvvZaKigqWLVvGM888w7Jly1LuZ9GiRcydO5clS5Ywf/58Fi5cWLfstNNOY+HChSxdupT999+f2267jXHjxnHqqafyy1/+kiVLluxwQt60aRPTpk3j3nvv5aWXXmLr1q3ccsstdct79+7N4sWLmTlzZmw1VqJb8MWLF3PvvffWPfUuuVvwpUuXctlllwGhW/ALL7yQpUuX8txzz9GvX7/MDioqQYhIljX0Sz+XEtVMEydOZO7cudx2220A3HfffZSXl7N161ZWrVrFq6++ysEHHxy7j7/97W9MmjSprsvtU089tW7Zyy+/zJVXXsmaNWvYsGEDJ5zQcBdxy5cvZ8iQIey7774ATJ06lZtvvplLLrkECAkHYPTo0fz5z3/eafuW0C14my9BZPvZuCJSGBMnTmTBggUsXryYjRs3Mnr0aN5++22uv/56FixYwLJlyzj55JNTdvPdmGnTpnHTTTfx0ksv8eMf/7jZ+0lIdBmeqrvwltAteJtOEIln477zDrhvfzaukoRI69OlSxeOOuoozj333LrG6XXr1tG5c2e6devGhx9+WFcFlcoRRxzBgw8+yOeff8769et56KGH6patX7+efv36sWXLFuYknSS6du3K+vXrd9rXsGHDqKqqorKyEgi9sh555JFpf56W0C14m04Q+Xo2rojkx+TJk1m6dGldghg+fDgjR45kv/3246yzzuLQQw9tcPtRo0ZxxhlnMHz4cE488US+/OUv1y275pprGDt2LIceeugODcpnnnkmv/zlLxk5cuQOz4vu1KkTd9xxB1/72tc46KCDKCoqYsaMGWl/lpbQLXjOuvvOt+Z0911UFEoO9ZmFfupFJD3q7rt1aEndfbd4qTqAVcewIiJtPEHk89m4IiKtTZtOEIV6Nq7IrmhXqa7eVTXn79Pm74OYMkUJQSRTnTp1YvXq1fTq1QszK3Q4Uo+7s3r16ibfH9HmE4SIZG7AgAFUV1dTU1NT6FAkhU6dOjFgwIAmbaMEISIZa9++PUOGDCl0GJJlbboNQkREUlOCEBGRWEoQIiISa5e5k9rMaoB3Ch1HA3oDHxc6iAYovswovswovsxkEt9gd+8Tt2CXSRAtnZlVpLqdvSVQfJlRfJlRfJnJVXyqYhIRkVhKECIiEksJIn/KCx1AIxRfZhRfZhRfZnISn9ogREQklkoQIiISSwlCRERiKUFkiZkNNLOnzOxVM3vFzL4Ts854M1trZkui4aoCxFllZi9F77/TI/gsuNHMKs1smZmNymNsw5KOzRIzW2dml9RbJ6/H0MxuN7OPzOzlpHk9zexxM3szeu2RYtup0TpvmtnUPMb3SzN7Pfr7/cXMuqfYtsHvQg7jm21mK5P+hiel2HaCmS2PvouX5zG+e5NiqzKzJSm2zcfxiz2v5O076O4asjAA/YBR0XhX4A3ggHrrjAf+p8BxVgG9G1h+EvAwYMBXgBcKFGcx8AHhJp6CHUPgCGAU8HLSvF8Al0fjlwPXxWzXE1gRvfaIxnvkKb7jgXbR+HVx8aXzXchhfLOBWWn8/d8C9gY6AEvr/z/lKr56y38FXFXA4xd7XsnXd1AliCxx91XuvjgaXw+8BvQvbFTNMhH4owf/BLqbWb8CxHEM8Ja7F/TueHd/Fvik3uyJwF3R+F3AV2M2PQF43N0/cfdPgceBCfmIz90fc/et0eQ/gab18ZxFKY5fOsYAle6+wt03A3MJxz2rGorPwoMtvg78V7bfN10NnFfy8h1UgsgBMysFRgIvxCw+xMyWmtnDZnZgfiMDwIHHzGyRmU2PWd4feC9puprCJLozSf2PWehjuIe7r4rGPwD2iFmnpRzHcwklwjiNfRdy6aKoCuz2FNUjLeH4HQ586O5vplie1+NX77ySl++gEkSWmVkX4AHgEndfV2/xYkKVyXDgN8CDeQ4P4DB3HwWcCFxoZkcUIIYGmVkH4FTg/pjFLeEY1vFQlm+R14qb2RXAVmBOilUK9V24BdgHGAGsIlTjtESTabj0kLfj19B5JZffQSWILDKz9oQ/4hx3/3P95e6+zt03ROPzgfZm1jufMbr7yuj1I+AvhKJ8spXAwKTpAdG8fDoRWOzuH9Zf0BKOIfBhototev0oZp2CHkczmwacAkyJTiA7SeO7kBPu/qG7b3P3WuDWFO9b6OPXDjgNuDfVOvk6finOK3n5DipBZElUX3kb8Jq7/2eKdfaM1sPMxhCO/+o8xtjZzLomxgmNmS/XW20ecI4FXwHWJhVl8yXlL7dCH8PIPCBxRchU4K8x6zwKHG9mPaIqlOOjeTlnZhOAy4BT3X1jinXS+S7kKr7kNq1JKd53ITDUzIZEJcozCcc9X44FXnf36riF+Tp+DZxX8vMdzGULfFsagMMIxbxlwJJoOAmYAcyI1rkIeIVwRcY/gXF5jnHv6L2XRnFcEc1PjtGAmwlXkLwElOU5xs6EE363pHkFO4aERLUK2EKowz0P6AUsAN4EngB6RuuWAX9I2vZcoDIavpnH+CoJdc+J7+HvonX3AuY39F3IU3x3R9+tZYQTXb/68UXTJxGu2nkrn/FF8+9MfOeS1i3E8Ut1XsnLd1BdbYiISCxVMYmISCwlCBERiaUEISIisZQgREQklhKEiIjEUoIQaYSZbbMde5nNWs+iZlaa3JOoSEvSrtABiLQCn7v7iEIHIZJvKkGINFP0PIBfRM8E+JeZfSmaX2pmT0ad0S0ws0HR/D0sPJ9haTSMi3ZVbGa3Rv39P2Zmu0XrXxw9B2CZmc0t0MeUNkwJQqRxu9WrYjojadladz8IuAm4IZr3G+Audz+Y0FHejdH8G4FnPHQ0OIpwBy7AUOBmdz8QWAP8ezT/cmBktJ8ZufloIqnpTmqRRpjZBnfvEjO/Cjja3VdEHap94O69zOxjQvcRW6L5q9y9t5nVAAPc/YukfZQS+uwfGk1/H2jv7j81s0eADYQeax/0qJNCkXxRCUIkM55ivCm+SBrfxva2wZMJ/WKNAhZGPYyK5I0ShEhmzkh6fT4af47Q+yjAFOBv0fgCYCaAmRWbWbdUOzWzImCguz8FfB/oBuxUihHJJf0iEWncbrbjg+sfcffEpa49zGwZoRQwOZr3beAOM/seUAN8M5r/HaDczM4jlBRmEnoSjVMM3BMlEQNudPc1Wfo8ImlRG4RIM0VtEGXu/nGhYxHJBVUxiYhILJUgREQklkoQIiISSwlCRERiKUGIiEgsJQgREYmlBCEiIrH+P7TXVFp+QWs2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 733 ms (started: 2021-07-22 17:05:35 +08:00)\n"
     ]
    }
   ],
   "source": [
    "plt.clf()\n",
    "acc = history_dict[\"accuracy\"]\n",
    "val_acc = history_dict[\"val_accuracy\"]\n",
    "plt.plot(epochs, acc, \"bo\", label=\"Training acc\")\n",
    "plt.plot(epochs, val_acc, \"b\", label=\"Validation acc\")\n",
    "plt.title(\"Training and validation accuracy\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "psychological-daniel",
   "metadata": {},
   "source": [
    "如您所见，训练损失随着每个 epoch 减少，而训练准确度随着每个 epoch 增加。 这就是您在运行梯度下降优化时所期望的——每次迭代时您试图最小化的数量应该更少。 但是塔\n",
    "验证损失和准确性并非如此：它们似乎在第四个时期达到顶峰。 这是我们之前警告过的一个例子：在训练数据上表现更好的模型不一定是在以前从未见过的数据上表现更好的模型。 准确地说，您所看到的是：在第四个时期之后，您对训练过度拟合数据进行了过度优化，并且最终学习了特定于训练数据的表示，并且不会泛化到外部数据 训练集。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "revolutionary-commerce",
   "metadata": {},
   "source": [
    "在这种情况下，为了防止过度拟合，您可以在四个 epoch 后停止训练。 一般来说，您可以使用一系列技术来减轻过拟合，我们将在第 5 章中介绍。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "determined-hamilton",
   "metadata": {},
   "source": [
    "让我们从头开始训练一个新模型四个时期，然后在测试数据上对其进行评估。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "typical-suspect",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/4\n",
      "49/49 [==============================] - 2s 19ms/step - loss: 0.4582 - accuracy: 0.8286\n",
      "Epoch 2/4\n",
      "49/49 [==============================] - 1s 19ms/step - loss: 0.2603 - accuracy: 0.9104\n",
      "Epoch 3/4\n",
      "49/49 [==============================] - 1s 15ms/step - loss: 0.2027 - accuracy: 0.9285\n",
      "Epoch 4/4\n",
      "49/49 [==============================] - 1s 15ms/step - loss: 0.1701 - accuracy: 0.9403\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f5784a67970>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 5.59 s (started: 2021-07-22 17:08:49 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(16, activation=\"relu\"),\n",
    "    layers.Dense(16, activation=\"relu\"),\n",
    "    layers.Dense(1, activation=\"sigmoid\")\n",
    "])\n",
    "\n",
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"binary_crossentropy\",\n",
    "              metrics=[\"accuracy\"])\n",
    "\n",
    "model.fit(x_train, y_train, epochs=4, batch_size=512)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "macro-spirit",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "782/782 [==============================] - 6s 7ms/step - loss: 0.2982 - accuracy: 0.8807\n",
      "time: 7.26 s (started: 2021-07-22 17:08:57 +08:00)\n"
     ]
    }
   ],
   "source": [
    "results = model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "compatible-strap",
   "metadata": {},
   "source": [
    "最终结果如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "grave-command",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.29824867844581604, 0.8806800246238708]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.86 ms (started: 2021-07-22 17:09:43 +08:00)\n"
     ]
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "taken-philadelphia",
   "metadata": {},
   "source": [
    "第一个数字 0.29 是测试损失，第二个数字 0.88 是测试精度。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "delayed-brooklyn",
   "metadata": {},
   "source": [
    "这种相当幼稚的方法达到了 88% 的准确率。 使用最先进的方法，您应该能够接近 95%。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "animated-network",
   "metadata": {},
   "source": [
    "### 使用经过训练的模型对新数据进行预测"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "improved-laptop",
   "metadata": {},
   "source": [
    "训练模型后，您将希望在实际环境中使用它。 您可以使用该方法生成正面评价的可能性，正如您在第 3 章预测中学到的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "fewer-donna",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.2246211 ],\n",
       "       [0.9999509 ],\n",
       "       [0.9513612 ],\n",
       "       ...,\n",
       "       [0.17931864],\n",
       "       [0.09724324],\n",
       "       [0.7526973 ]], dtype=float32)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4.12 s (started: 2021-07-22 17:12:19 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "friendly-passport",
   "metadata": {},
   "source": [
    "如您所见，该模型对某些样本（0.99 或更多，或 0.01 或更少）有信心，但对其他样本（0.6、0.4）的信心较低。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "helpful-paintball",
   "metadata": {},
   "source": [
    "### 进一步的实验"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "removable-response",
   "metadata": {},
   "source": [
    "以下实验将帮助您相信您所做的架构选择都是相当合理的，尽管仍有改进的空间：\n",
    "* 您在最终分类层之前使用了两个表示层。 尝试使用一到三个表示层，看看这样做会如何影响验证和测试准确性。\n",
    "* 尝试使用具有更多单位或更少单位的图层：32 个单位、64 个单位等。\n",
    "* 尝试使用mse损失函数而不是binary_crossentropy。\n",
    "* 尝试使用tanh激活（一种在神经网络早期流行的激活）而不是 relu。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "varied-metropolitan",
   "metadata": {},
   "source": [
    "### 包起来"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "innovative-debut",
   "metadata": {},
   "source": [
    "以下是你应该从这个例子中学到的东西：\n",
    "* 您通常需要对原始数据进行大量预处理，以便能够将其作为张量输入到神经网络中。单词序列可以编码为二进制向量，但也有其他编码选项。\n",
    "* 带有激活的堆叠层可以解决广泛的问题（包括情感分类），您可能会经常使用它们。\n",
    "* 在二元分类问题（两个输出类）中，您的模型应该以一个带有一个单元和一个激活的 Dense 层结束：模型的输出应该是 sigmoid 一个介于 0 和 1 之间的标量，编码概率。\n",
    "* 在二元分类问题上有这样一个标量 sigmoid 输出，你应该使用的损失函数是 binary_crossentropy。\n",
    "* rmsprop 优化器通常是一个足够好的选择，无论您遇到什么问题。这是您不必担心的一件事。\n",
    "* 随着他们在训练数据上变得更好，神经网络最终开始过度拟合，并最终在他们以前从未见过的数据上获得越来越糟糕的结果。确保始终监控训练集之外数据的性能。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "alternate-carpet",
   "metadata": {},
   "source": [
    "## 新闻专线分类：多类分类示例"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "thorough-young",
   "metadata": {},
   "source": [
    "在上一节中，您看到了如何使用密集连接的神经网络将向量输入分类为两个互斥的类别。 但是当你有两个以上的类时会发生什么？\n",
    "\n",
    "在本节中，您将构建一个模型，将 Reuters 新闻专线分为 46 个相互排斥的主题。 因为你有很多类，这个问题是多类分类的一个实例； 并且因为每个数据点应该只归入一个类别，所以问题更具体地说是单标签多类分类的一个实例。 如果每个数据点可以属于多个类别（在本例中为主题），您将面临多标签多类别分类问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "communist-romance",
   "metadata": {},
   "source": [
    "### 路透社数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "renewable-qatar",
   "metadata": {},
   "source": [
    "您将使用 Reuters 数据集，这是一组简短的新闻专线及其主题，由 Reuters 于 1986 年发布。这是一个简单、广泛使用的文本分类玩具数据集。 有46个不同的主题； 有些主题比其他主题更具代表性，但每个主题在训练集中至少有 10 个示例。\n",
    "\n",
    "与 IMDB 和 MNIST 一样，路透社数据集打包为 Keras 的一部分。 让我们来看看。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "legendary-iceland",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 768 ms (started: 2021-07-23 14:24:27 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import reuters\n",
    "\n",
    "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(num_words=10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "swiss-leader",
   "metadata": {},
   "source": [
    "与 IMDB 数据集一样，参数 num_words=10000 将数据限制为数据中最常出现的 10,000 个单词。 \n",
    "\n",
    "您有 8,982 个训练示例和 2,246 个测试示例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "average-promotion",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8982"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.24 ms (started: 2021-07-23 14:24:31 +08:00)\n"
     ]
    }
   ],
   "source": [
    "len(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "subject-despite",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2246"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.58 ms (started: 2021-07-23 14:24:33 +08:00)\n"
     ]
    }
   ],
   "source": [
    "len(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "revised-gossip",
   "metadata": {},
   "source": [
    "与 IMDB 评论一样，每个示例都是一个整数列表（单词索引）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "efficient-embassy",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1,\n",
       " 245,\n",
       " 273,\n",
       " 207,\n",
       " 156,\n",
       " 53,\n",
       " 74,\n",
       " 160,\n",
       " 26,\n",
       " 14,\n",
       " 46,\n",
       " 296,\n",
       " 26,\n",
       " 39,\n",
       " 74,\n",
       " 2979,\n",
       " 3554,\n",
       " 14,\n",
       " 46,\n",
       " 4689,\n",
       " 4329,\n",
       " 86,\n",
       " 61,\n",
       " 3499,\n",
       " 4795,\n",
       " 14,\n",
       " 61,\n",
       " 451,\n",
       " 4329,\n",
       " 17,\n",
       " 12]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.26 ms (started: 2021-07-23 14:24:35 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_data[10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "binding-jersey",
   "metadata": {},
   "source": [
    "如果您感到好奇，您可以通过以下方法将其解码为文字。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "united-publicity",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 221 ms (started: 2021-07-23 14:24:38 +08:00)\n"
     ]
    }
   ],
   "source": [
    "word_index = reuters.get_word_index()\n",
    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
    "decoded_newswire = \" \".join([reverse_word_index.get(i - 3, \"?\") for i in train_data[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "conservative-cedar",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'? ? ? said as a result of its december acquisition of space co it expects earnings per share in 1987 of 1 15 to 1 30 dlrs per share up from 70 cts in 1986 the company said pretax net should rise to nine to 10 mln dlrs from six mln dlrs in 1986 and rental operation revenues to 19 to 22 mln dlrs from 12 5 mln dlrs it said cash flow per share this year should be 2 50 to three dlrs reuter 3'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.68 ms (started: 2021-07-23 14:24:40 +08:00)\n"
     ]
    }
   ],
   "source": [
    "decoded_newswire"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "gentle-connection",
   "metadata": {},
   "source": [
    "> 请注意，索引偏移了 3，因为 0、1 和 2 是“填充”、“序列开始”和“未知”的保留索引。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "affected-housing",
   "metadata": {},
   "source": [
    "与示例关联的标签是 0 到 45 之间的整数——主题索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "detected-edward",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.07 ms (started: 2021-07-23 14:24:42 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_labels[10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "agreed-plate",
   "metadata": {},
   "source": [
    "### 准备数据"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "outer-canvas",
   "metadata": {},
   "source": [
    "您可以使用与前一个示例完全相同的代码对数据进行矢量化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "least-entrepreneur",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 509 ms (started: 2021-07-23 14:25:16 +08:00)\n"
     ]
    }
   ],
   "source": [
    "x_train = vectorize_sequences(train_data)\n",
    "x_test = vectorize_sequences(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "coral-airport",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8982, 10000)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.6 ms (started: 2021-07-23 14:25:29 +08:00)\n"
     ]
    }
   ],
   "source": [
    "x_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "amateur-watson",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8982,)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.17 ms (started: 2021-07-23 14:25:50 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "athletic-martin",
   "metadata": {},
   "source": [
    "要矢量化标签，有两种可能性：您可以将标签列表转换为整数张量，或者您可以使用单热编码。 One-hot encoding 是一种广泛使用的分类数据格式，也称为分类编码。 有关 one-hot 编码的更详细说明，请参阅第 4.1.2 节。 在这种情况下，标签的单热编码包括将每个标签嵌入为一个全零向量，标签索引位置为 1。 下面是一个例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "atlantic-valuation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4.59 ms (started: 2021-07-23 14:28:00 +08:00)\n"
     ]
    }
   ],
   "source": [
    "def to_one_hot(labels, dimension=46):\n",
    "    results = np.zeros((len(labels), dimension))\n",
    "    for i, label in enumerate(labels):\n",
    "        results[i, label] = 1.\n",
    "    return results\n",
    "\n",
    "y_train = to_one_hot(train_labels)\n",
    "y_test = to_one_hot(test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prescription-modification",
   "metadata": {},
   "source": [
    "请注意，在 Keras 中有一种内置的方法可以做到这一点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "robust-kitchen",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.63 ms (started: 2021-07-23 14:30:49 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.utils import to_categorical\n",
    "y_train = to_categorical(train_labels)\n",
    "y_test = to_categorical(test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "lucky-karma",
   "metadata": {},
   "source": [
    "### 构建模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cloudy-great",
   "metadata": {},
   "source": [
    "这个主题分类问题看起来类似于之前的电影评论分类问题：在这两种情况下，您都在尝试对短文本片段进行分类。 但是这里有一个新的约束：输出类的数量从 2 到 46。输出空间的维度要大得多。\n",
    "\n",
    "在您一直使用的一堆层中，每一层只能访问存在于前一层输出中的密集信息。 如果一层丢掉了一些与分类问题相关的信息，那么后面的层就永远无法恢复这些信息：每一层都可能成为信息瓶颈。 在前面的示例中，您使用了 16 维中间层，但 16 维空间可能太有限，无法学习分离 46 个不同的类：这样的小层可能会成为信息瓶颈，永久丢弃相关信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dominant-investigation",
   "metadata": {},
   "source": [
    "出于这个原因，您将使用更大的层。 让我们用 64 个单位。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "global-tournament",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 541 µs (started: 2021-07-23 14:39:02 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "chinese-teddy",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.55 s (started: 2021-07-23 14:39:03 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(46, activation=\"softmax\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "needed-yeast",
   "metadata": {},
   "source": [
    "关于此架构，您还应该注意另外两件事：\n",
    "\n",
    "您以大小为 46 的层结束模型。这意味着对于每个输入样本，网络 Dense 将输出一个 46 维向量。 此向量中的每个条目（每个维度）都将编码不同的输出类。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "varied-bones",
   "metadata": {},
   "source": [
    "最后一层使用激活。 您在 MNIST 示例中看到了这种模式。 这意味着 softmax 模型将输出 46 个不同输出类的概率分布——对于每个输入样本，模型将产生一个 46 维的输出向量，其中 是样本属于第 i 类的 output[i] 概率。 46 分总和为 1。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eligible-dayton",
   "metadata": {},
   "source": [
    "在这种情况下使用的最佳损失函数是 categorical_crossentropy。 它测量两个概率分布之间的距离：这里是模型输出的概率分布与标签的真实分布之间的距离。 通过最小化这两个分布之间的距离，您可以训练模型输出尽可能接近真实标签的内容。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "ethical-insured",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 13.5 ms (started: 2021-07-23 14:54:36 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "occupied-disaster",
   "metadata": {},
   "source": [
    "### 验证您的方法"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "signal-track",
   "metadata": {},
   "source": [
    "让我们在训练数据中分离出 1,000 个样本用作验证集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "consistent-medium",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.49 ms (started: 2021-07-23 15:50:28 +08:00)\n"
     ]
    }
   ],
   "source": [
    "x_val = x_train[:1000]\n",
    "partial_x_train = x_train[1000:]\n",
    "y_val = y_train[:1000]\n",
    "partial_y_train = y_train[1000:]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "infectious-office",
   "metadata": {},
   "source": [
    "现在，让我们训练模型 20 个周期。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "incorporate-german",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "16/16 [==============================] - 3s 46ms/step - loss: 2.6299 - accuracy: 0.5482 - val_loss: 1.7317 - val_accuracy: 0.6480\n",
      "Epoch 2/20\n",
      "16/16 [==============================] - 0s 25ms/step - loss: 1.3951 - accuracy: 0.7085 - val_loss: 1.3009 - val_accuracy: 0.7100\n",
      "Epoch 3/20\n",
      "16/16 [==============================] - 0s 25ms/step - loss: 1.0199 - accuracy: 0.7820 - val_loss: 1.1186 - val_accuracy: 0.7630\n",
      "Epoch 4/20\n",
      "16/16 [==============================] - 0s 26ms/step - loss: 0.7993 - accuracy: 0.8295 - val_loss: 1.0542 - val_accuracy: 0.7670\n",
      "Epoch 5/20\n",
      "16/16 [==============================] - 0s 29ms/step - loss: 0.6398 - accuracy: 0.8629 - val_loss: 0.9707 - val_accuracy: 0.8100\n",
      "Epoch 6/20\n",
      "16/16 [==============================] - 0s 27ms/step - loss: 0.5150 - accuracy: 0.8915 - val_loss: 0.9248 - val_accuracy: 0.8110\n",
      "Epoch 7/20\n",
      "16/16 [==============================] - 0s 26ms/step - loss: 0.4156 - accuracy: 0.9136 - val_loss: 0.9284 - val_accuracy: 0.8140\n",
      "Epoch 8/20\n",
      "16/16 [==============================] - 0s 23ms/step - loss: 0.3416 - accuracy: 0.9270 - val_loss: 0.9294 - val_accuracy: 0.8040\n",
      "Epoch 9/20\n",
      "16/16 [==============================] - 0s 23ms/step - loss: 0.2814 - accuracy: 0.9375 - val_loss: 0.9317 - val_accuracy: 0.8050\n",
      "Epoch 10/20\n",
      "16/16 [==============================] - 0s 23ms/step - loss: 0.2384 - accuracy: 0.9445 - val_loss: 0.9568 - val_accuracy: 0.8200\n",
      "Epoch 11/20\n",
      "16/16 [==============================] - 0s 24ms/step - loss: 0.2084 - accuracy: 0.9489 - val_loss: 0.9440 - val_accuracy: 0.8190\n",
      "Epoch 12/20\n",
      "16/16 [==============================] - 0s 25ms/step - loss: 0.1821 - accuracy: 0.9521 - val_loss: 0.9730 - val_accuracy: 0.8000\n",
      "Epoch 13/20\n",
      "16/16 [==============================] - 0s 23ms/step - loss: 0.1688 - accuracy: 0.9515 - val_loss: 0.9698 - val_accuracy: 0.8140\n",
      "Epoch 14/20\n",
      "16/16 [==============================] - 0s 24ms/step - loss: 0.1496 - accuracy: 0.9546 - val_loss: 0.9929 - val_accuracy: 0.8140\n",
      "Epoch 15/20\n",
      "16/16 [==============================] - 0s 23ms/step - loss: 0.1403 - accuracy: 0.9558 - val_loss: 1.0212 - val_accuracy: 0.8070\n",
      "Epoch 16/20\n",
      "16/16 [==============================] - 0s 23ms/step - loss: 0.1309 - accuracy: 0.9565 - val_loss: 1.0357 - val_accuracy: 0.8120\n",
      "Epoch 17/20\n",
      "16/16 [==============================] - 0s 24ms/step - loss: 0.1274 - accuracy: 0.9562 - val_loss: 1.0408 - val_accuracy: 0.8080\n",
      "Epoch 18/20\n",
      "16/16 [==============================] - 0s 27ms/step - loss: 0.1225 - accuracy: 0.9559 - val_loss: 1.0483 - val_accuracy: 0.8060\n",
      "Epoch 19/20\n",
      "16/16 [==============================] - 0s 27ms/step - loss: 0.1138 - accuracy: 0.9575 - val_loss: 1.0904 - val_accuracy: 0.8010\n",
      "Epoch 20/20\n",
      "16/16 [==============================] - 0s 27ms/step - loss: 0.1145 - accuracy: 0.9579 - val_loss: 1.1013 - val_accuracy: 0.8110\n",
      "time: 12.1 s (started: 2021-07-23 15:52:28 +08:00)\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(partial_x_train,\n",
    "                    partial_y_train,\n",
    "                    epochs=20,\n",
    "                    batch_size=512,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "alike-hartford",
   "metadata": {},
   "source": [
    "最后，让我们展示一下它的损失和准确度曲线（见图 4.6 和 4.7）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "fancy-cocktail",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 540 ms (started: 2021-07-23 15:54:02 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "endangered-uncertainty",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqf0lEQVR4nO3deZwU1bn/8c8DjA4Ioiwq6wxERUVhgAEUokHNL3EhIkajXK5KiCJcE7e4kHCj/Ey4NyYm1xC3oEaNkqCJkesad4J7BIIoij9BQUFUHBWGsAj4/P44NdAM3TM9S3X1TH/fr1e9uvZ+uqannj7nVJ0yd0dERApXi6QDEBGRZCkRiIgUOCUCEZECp0QgIlLglAhERAqcEoGISIFTIpBGZWaPmtnZjb1uksxsuZl9PYb9upntH43fbGY/yWbderzPWDN7vL5x1rDfEWa2srH3K7nXKukAJHlmtj5lsg2wGdgWTZ/n7jOz3Ze7Hx/Hus2du09sjP2YWSnwLlDk7lujfc8Esv4bSuFRIhDcvW3VuJktB85x9yerr2dmrapOLiLSfKhqSDKqKvqb2RVm9iFwu5ntbWYPmdkaM/ssGu+ess0cMzsnGh9nZs+Z2bXRuu+a2fH1XLeXmc01s0oze9LMbjCzuzPEnU2MPzWz56P9PW5mnVKWn2lmK8yswsym1HB8hprZh2bWMmXeaDNbFI0PMbMXzexzM1ttZteb2W4Z9nWHmf0sZfqyaJsPzGx8tXVPNLN/mtk6M3vfzKamLJ4bvX5uZuvN7IiqY5uy/TAze8XM1kavw7I9NjUxs4Oj7T83s8VmdlLKshPM7I1on6vM7NJofqfo7/O5mX1qZs+amc5LOaYDLrXZD+gAlAATCN+Z26PpnsBG4Poath8KvAV0An4B3GZmVo91/wj8A+gITAXOrOE9s4nx34DvAvsAuwFVJ6ZDgJui/XeN3q87abj7y8C/gGOq7feP0fg24OLo8xwBHAv8Rw1xE8VwXBTP/wEOAKq3T/wLOAvYCzgRmGRmJ0fLjope93L3tu7+YrV9dwAeBqZHn+3XwMNm1rHaZ9jl2NQScxHwIPB4tN0PgJlm1ida5TZCNWM74FDg6Wj+D4GVQGdgX+DHgPq9yTElAqnNl8BV7r7Z3Te6e4W73+fuG9y9EpgGfK2G7Ve4+y3uvg24E+hC+IfPel0z6wkMBq509y/c/TnggUxvmGWMt7v7/3P3jcC9QFk0/1TgIXef6+6bgZ9ExyCTPwFjAMysHXBCNA93n+/uL7n7VndfDvwuTRzpfCeK73V3/xch8aV+vjnu/pq7f+nui6L3y2a/EBLH2+5+VxTXn4AlwLdS1sl0bGpyONAW+Hn0N3oaeIjo2ABbgEPMbE93/8zdF6TM7wKUuPsWd3/W1QFazikRSG3WuPumqgkza2Nmv4uqTtYRqiL2Sq0eqebDqhF33xCNtq3jul2BT1PmAbyfKeAsY/wwZXxDSkxdU/cdnYgrMr0X4df/KWa2O3AKsMDdV0RxHBhVe3wYxfFfhNJBbXaKAVhR7fMNNbNnoqqvtcDELPdbte8V1eatALqlTGc6NrXG7O6pSTN1v98mJMkVZvZ3Mzsimv9LYCnwuJm9Y2aTs/sY0piUCKQ21X+d/RDoAwx19z3ZURWRqbqnMawGOphZm5R5PWpYvyExrk7dd/SeHTOt7O5vEE54x7NztRCEKqYlwAFRHD+uTwyE6q1UfySUiHq4e3vg5pT91vZr+gNClVmqnsCqLOKqbb89qtXvb9+vu7/i7qMI1UazCSUN3L3S3X/o7r2Bk4BLzOzYBsYidaREIHXVjlDn/nlU33xV3G8Y/cKeB0w1s92iX5PfqmGThsT4F2CkmX01ati9mtr/T/4IXEhIOH+uFsc6YL2ZHQRMyjKGe4FxZnZIlIiqx9+OUELaZGZDCAmoyhpCVVbvDPt+BDjQzP7NzFqZ2enAIYRqnIZ4mVB6uNzMisxsBOFvNCv6m401s/buvoVwTL4EMLORZrZ/1Ba0ltCuUlNVnMRAiUDq6jqgNfAJ8BLwtxy971hCg2sF8DPgHsL9DulcRz1jdPfFwPmEk/tq4DNCY2ZNquron3b3T1LmX0o4SVcCt0QxZxPDo9FneJpQbfJ0tVX+A7jazCqBK4l+XUfbbiC0iTwfXYlzeLV9VwAjCaWmCuByYGS1uOvM3b8gnPiPJxz3G4Gz3H1JtMqZwPKoimwi4e8JoTH8SWA98CJwo7s/05BYpO5M7TLSFJnZPcASd4+9RCLS3KlEIE2CmQ02s6+YWYvo8spRhLpmEWkg3VksTcV+wF8JDbcrgUnu/s9kQxJpHlQ1JCJS4FQ1JCJS4Jpc1VCnTp28tLQ06TBERJqU+fPnf+LundMta3KJoLS0lHnz5iUdhohIk2Jm1e8o305VQyIiBU6JQESkwCkRiIgUuCbXRiAiubdlyxZWrlzJpk2bal9ZElVcXEz37t0pKirKehslAhGp1cqVK2nXrh2lpaVkfq6QJM3dqaioYOXKlfTq1Svr7QqiamjmTCgthRYtwutMPcZbpE42bdpEx44dlQTynJnRsWPHOpfcmn2JYOZMmDABNkSPNFmxIkwDjB2beTsR2ZmSQNNQn79Tsy8RTJmyIwlU2bAhzBcRkQJIBO+9V7f5IpJ/KioqKCsro6ysjP32249u3bptn/7iiy9q3HbevHlccMEFtb7HsGHDGiXWOXPmMHLkyEbZV640+0TQs/pD/mqZLyIN19jtch07dmThwoUsXLiQiRMncvHFF2+f3m233di6dWvGbcvLy5k+fXqt7/HCCy80LMgmrNkngmnToE2bnee1aRPmi0jjq2qXW7EC3He0yzX2RRrjxo1j4sSJDB06lMsvv5x//OMfHHHEEQwYMIBhw4bx1ltvATv/Qp86dSrjx49nxIgR9O7de6cE0bZt2+3rjxgxglNPPZWDDjqIsWPHUtVL8yOPPMJBBx3EoEGDuOCCC2r95f/pp59y8skn069fPw4//HAWLVoEwN///vftJZoBAwZQWVnJ6tWrOeqooygrK+PQQw/l2WefbdwDVoNm31hc1SA8ZUqoDurZMyQBNRSLxKOmdrnG/r9buXIlL7zwAi1btmTdunU8++yztGrViieffJIf//jH3Hfffbtss2TJEp555hkqKyvp06cPkyZN2uWa+3/+858sXryYrl27Mnz4cJ5//nnKy8s577zzmDt3Lr169WLMmDG1xnfVVVcxYMAAZs+ezdNPP81ZZ53FwoULufbaa7nhhhsYPnw469evp7i4mBkzZvDNb36TKVOmsG3bNjZUP4gxavaJAMKXTyd+kdzIZbvcaaedRsuWLQFYu3YtZ599Nm+//TZmxpYtW9Juc+KJJ7L77ruz++67s88++/DRRx/RvXv3ndYZMmTI9nllZWUsX76ctm3b0rt37+3X548ZM4YZM2bUGN9zzz23PRkdc8wxVFRUsG7dOoYPH84ll1zC2LFjOeWUU+jevTuDBw9m/PjxbNmyhZNPPpmysrKGHJo6afZVQyKSW7lsl9tjjz22j//kJz/h6KOP5vXXX+fBBx/MeC397rvvvn28ZcuWadsXslmnISZPnsytt97Kxo0bGT58OEuWLOGoo45i7ty5dOvWjXHjxvGHP/yhUd+zJkoEItKokmqXW7t2Ld26dQPgjjvuaPT99+nTh3feeYfly5cDcM8999S6zZFHHsnMqHFkzpw5dOrUiT333JNly5Zx2GGHccUVVzB48GCWLFnCihUr2HfffTn33HM555xzWLBgQaN/hkxiSwRm1sPMnjGzN8xssZldmGadEWa21swWRsOVccUjIrkxdizMmAElJWAWXmfMiL969vLLL+dHP/oRAwYMaPRf8ACtW7fmxhtv5LjjjmPQoEG0a9eO9u3b17jN1KlTmT9/Pv369WPy5MnceeedAFx33XUceuih9OvXj6KiIo4//njmzJlD//79GTBgAPfccw8XXrjLKTM2sT2z2My6AF3cfYGZtQPmAye7+xsp64wALnX3rC+6LS8vdz2YRiS33nzzTQ4++OCkw0jc+vXradu2Le7O+eefzwEHHMDFF1+cdFi7SPf3MrP57l6ebv3YSgTuvtrdF0TjlcCbQLe43k9EJG633HILZWVl9O3bl7Vr13LeeeclHVKjyMlVQ2ZWCgwAXk6z+AgzexX4gFA6WJxm+wnABICeuhNMRBJy8cUX52UJoKFibyw2s7bAfcBF7r6u2uIFQIm79wd+C8xOtw93n+Hu5e5e3rlz2mcvi4hIPcWaCMysiJAEZrr7X6svd/d17r4+Gn8EKDKzTnHGJCIiO4vzqiEDbgPedPdfZ1hnv2g9zGxIFE9FXDGJiMiu4mwjGA6cCbxmZgujeT8GegK4+83AqcAkM9sKbATO8LguYxIRkbTivGroOXc3d+/n7mXR8Ii73xwlAdz9enfv6+793f1wdy/c7v9EJKOjjz6axx57bKd51113HZMmTcq4zYgRI6i61PyEE07g888/32WdqVOncu2119b43rNnz+aNN7Zf9c6VV17Jk08+WYfo08un7qp1Z7GI5L0xY8Ywa9asnebNmjUrq47fIPQautdee9Xrvasngquvvpqvf/3r9dpXvlIiEJG8d+qpp/Lwww9vfwjN8uXL+eCDDzjyyCOZNGkS5eXl9O3bl6uuuirt9qWlpXzyyScATJs2jQMPPJCvfvWr27uqhnCPwODBg+nfvz/f/va32bBhAy+88AIPPPAAl112GWVlZSxbtoxx48bxl7/8BYCnnnqKAQMGcNhhhzF+/Hg2b968/f2uuuoqBg4cyGGHHcaSJUtq/HxJd1ddEL2PikjjuegiWLiwcfdZVgbXXZd5eYcOHRgyZAiPPvooo0aNYtasWXznO9/BzJg2bRodOnRg27ZtHHvssSxatIh+/fql3c/8+fOZNWsWCxcuZOvWrQwcOJBBgwYBcMopp3DuuecC8J//+Z/cdttt/OAHP+Ckk05i5MiRnHrqqTvta9OmTYwbN46nnnqKAw88kLPOOoubbrqJiy66CIBOnTqxYMECbrzxRq699lpuvfXWjJ8v6e6qVSIQkSYhtXootVro3nvvZeDAgQwYMIDFixfvVI1T3bPPPsvo0aNp06YNe+65JyeddNL2Za+//jpHHnkkhx12GDNnzmTx4l3ubd3JW2+9Ra9evTjwwAMBOPvss5k7d+725aeccgoAgwYN2t5RXSbPPfccZ555JpC+u+rp06fz+eef06pVKwYPHsztt9/O1KlTee2112jXrl2N+86GSgQiUic1/XKP06hRo7j44otZsGABGzZsYNCgQbz77rtce+21vPLKK+y9996MGzcuY/fTtRk3bhyzZ8+mf//+3HHHHcyZM6dB8VZ1Zd2QbqwnT57MiSeeyCOPPMLw4cN57LHHtndX/fDDDzNu3DguueQSzjrrrAbFqhKBiDQJbdu25eijj2b8+PHbSwPr1q1jjz32oH379nz00Uc8+uijNe7jqKOOYvbs2WzcuJHKykoefPDB7csqKyvp0qULW7Zs2d51NEC7du2orKzcZV99+vRh+fLlLF26FIC77rqLr33ta/X6bEl3V60SgYg0GWPGjGH06NHbq4iqum0+6KCD6NGjB8OHD69x+4EDB3L66afTv39/9tlnHwYPHrx92U9/+lOGDh1K586dGTp06PaT/xlnnMG5557L9OnTtzcSAxQXF3P77bdz2mmnsXXrVgYPHszEiRPr9bmqnqXcr18/2rRps1N31c888wwtWrSgb9++HH/88cyaNYtf/vKXFBUV0bZt20Z5gE1s3VDHRd1Qi+SeuqFuWvKmG2oREWkalAhERAqcEoGIZKWpVSMXqvr8nZQIRKRWxcXFVFRUKBnkOXenoqKC4uLiOm2nq4ZEpFbdu3dn5cqVrFmzJulQpBbFxcV07969TtsoEYhIrYqKiujVq1fSYUhMVDUkIlLglAhERAqcEoGISIFTIhARKXBKBCIiBU6JQESkwCkRiIgUOCUCEZECp0QgIlLglAhERAqcEoGISIFTIhARKXBKBCIiBU6JQESkwCkRiIgUOCUCEZECp0QgIlLgYksEZtbDzJ4xszfMbLGZXZhmHTOz6Wa21MwWmdnAuOIREZH04nxU5Vbgh+6+wMzaAfPN7Al3fyNlneOBA6JhKHBT9CoiIjkSW4nA3Ve7+4JovBJ4E+hWbbVRwB88eAnYy8y6xBWTiIjsKidtBGZWCgwAXq62qBvwfsr0SnZNFpjZBDObZ2bz1qxZE1ucIiKFKPZEYGZtgfuAi9x9XX324e4z3L3c3cs7d+7cuAGKiBS4WBOBmRURksBMd/9rmlVWAT1SprtH80REJEfivGrIgNuAN9391xlWewA4K7p66HBgrbuvjismERHZVZxXDQ0HzgReM7OF0bwfAz0B3P1m4BHgBGApsAH4bozxiIhIGrElAnd/DrBa1nHg/LhiEBGR2unOYhGRAqdEICJS4JQIREQKnBKBiEiBUyIQESlwSgQiIgVOiUBEpMApEYiIFDglAhGRAqdEICJS4JQIREQKnBKBiEiBUyIQESlwBZUIKiuTjkBEJP8UTCL485+ha1dYsSLpSERE8kvBJILDD4fNm+Haa5OOREQkvxRMIujRA/793+HWW+Hjj5OORkQkfxRMIgC44opQKvjNb5KOREQkfxRUIujTB045BW64AdauTToaEZH8UFCJAOBHPwpJ4Kabko5ERCQ/FFwiGDQIvvEN+J//gY0bk45GRCR5BZcIIJQKPv4Ybr896UhERJJXkInga18Ll5P+8pewdWvS0YiIJKsgE4FZKBUsXw6zZiUdjYhIsgoyEQCMHAl9+8LPfw5ffpl0NCIiySnYRNCiRSgVLF4MDz6YdDQiIskp2EQAcPrp0KsX/Pd/g3vS0YiIJKOgE0GrVnDZZfDyyzBnTtLRiIgko6ATAcB3vwv77htKBSIihajgE0FxMVx8MTzxBMybl3Q0IiK5F1siMLPfm9nHZvZ6huUjzGytmS2MhivjiqU2kyZB+/YqFYhIYYqzRHAHcFwt6zzr7mXRcHWMsdRozz3h+9+H+++HJUuSikJEJBmxJQJ3nwt8Gtf+G9uFF4ZqomuuSToSEZHcSrqN4Agze9XMHjWzvplWMrMJZjbPzOatWbMmlkA6d4ZzzoG774b33ovlLURE8lKSiWABUOLu/YHfArMzrejuM9y93N3LO3fuHFtAl14aXn/1q9jeQkQk72SVCMxsDzNrEY0faGYnmVlRQ97Y3de5+/po/BGgyMw6NWSfDdWzJ4wdC7fcAjEVPERE8k62JYK5QLGZdQMeB84kNAbXm5ntZ2YWjQ+JYqloyD4bwxVXwKZNepyliBSObBOBufsG4BTgRnc/DchYpw9gZn8CXgT6mNlKM/uemU00s4nRKqcCr5vZq8B04Az35Dt6OPhgGD0arr8e1q1LOhoRkfhlnQjM7AhgLPBwNK9lTRu4+xh37+LuRe7e3d1vc/eb3f3maPn17t7X3fu7++Hu/kL9P0bjqnqc5c03h+mZM6G0NHRUV1oapkVEmotWWa53EfAj4H53X2xmvYFnYosqYeXl8PWvh8dZdu4c7jHYsCEsW7ECJkwI42PHJhejiEhjsbrWxkSNxm3dPZGKk/Lycp+Xg74gnn4ajj0WOnSAT9PcDVFSEh5sIyLSFJjZfHcvT7cs26uG/mhme5rZHsDrwBtmdlljBplvjj4ahgxJnwRA9xqISPORbRvBIVEJ4GTgUaAX4cqhZqvqcZaZ9OyZu1hEROKUbSIoiu4bOBl4wN23AIlf4RO3k06Cbt1CUkjVpg1Mm5ZMTCIijS3bRPA7YDmwBzDXzEqAZn9xZYsWO55e1rlzSAglJTBjhhqKRaT5qHNj8fYNzVq5+9ZGjqdWuWosrrJlCxxwAHTtCs8/v2vpQESkKWiMxuL2Zvbrqo7fzOxXhNJBs1dUFB5n+eKLMHdu0tGIiDS+bKuGfg9UAt+JhnXA7XEFlW/Gj4d99oH/+q+kIxERaXzZJoKvuPtV7v5ONPxfoHecgeWT1q3hoovg8cd33G0sItJcZJsINprZV6smzGw4sDGekPLTRRfBCSeEx1r+5CehAVlEpDnItouJicAfzKx9NP0ZcHY8IeWn1q3hf/8XzjsPfvYz+OCDUDooalBn3CIiycsqEbj7q0B/M9szml5nZhcBi2KMLe+0agW33hruLfjpT+HDD+Hee2GPgmg2F5Hmqk5PKIseJlN1/8AlMcST98zg6qtDaeBvf4NjjtFDbESkaWvIoyoL+or6886Dv/4VFi2C4cPhnXeSjkhEpH4akggKvrl01Ch46imoqIAjjoD585OOSESk7mpMBGZWaWbr0gyVQNccxZjXhg2D556D4mIYMSJcYioi0pTUmAjcvZ2775lmaOfu2V5x1OwdfHC487h3bzjxRLjrrqQjEhHJXkOqhiRF166hC4ojj4SzzoJrrtG9BiLSNCgRNKL27eHRR+GMM2DyZLjwQti2LemoRERqpuqdRrb77uHh9l27wq9/DatXh6qi4uKkIxMRSU+JIAYtWsCvfhWSwaWXhvsMZs+GvfZKOjIRkV2paihGP/wh/PGP8MILoe1g5cqkIxIR2ZUSQczGjAntBitWwNChoZ+ixYvVkCwi+aPeTyhLSq6fUNZYFi6E888PpQMITz0bPRpOPjkkiBZKySIF74svYN06qKwMr1VD1fRhh4WbV+ujpieUqY0gR8rKwqMuP/gg9GJ6//2hMfkXv4AuXcJdyqNHh5vSdtst6WhFpLGsWgWvvAILFsDHH+96ck+d3ry55n1demn9E0FNVCJI0GefwcMPh6Twt7/Bhg3hEtSRI0NSOO449Wwq0pRUVISTftUwb164chCgZUvo1An23DMM7drVPl59eu+9Q5f49VFTiUCJIE9s3AhPPBGSwgMPwKefhktOv/GNkBS+9S3o2DHpKEWkSmVl+JWfeuJ/992wzAz69IHBg6G8PLyWldX/JN4YlAiamK1b4dlnQ1K4//5wtVHLlnDUUfC978Gpp4b7FUSk8W3ZEkrnGzeGoWp8/Xp4/fUdJ/0lS3Zc9FFSEk72VcOgQeEXfD5RIkjYzJkwZQq89x707AnTpsHYsdlt6x56Nb3/frjnHli2LBQvx4+HCRPgK1+JN3aRJG3cGErHVcPGjaFBdfPm9K81Ldu8GTZtSn+STx2vrTeAfffd+aRfXg6dO+fmeDREIonAzH4PjAQ+dvdD0yw34DfACcAGYJy7L6htv00tEcycGU7YGzbsmNemDcyYkX0yqPLll6Hb65tuCtVH27bBN78ZnqN84onhCWoi+cg9PNFvzZodJ/WKip1P8ummN22q3/vtvnu46GK33XYeb906/P+1bp3dePV5Bx4I3buHqp+mJqlEcBSwHvhDhkRwAvADQiIYCvzG3YfWtt+mlghKS8M9BNWVlMDy5fXf76pVcMstYfjgg/DlnDAhVB11VQfhkoCtW0Opd9kyWLp059dly8Kv7XR22y20f3XoEIbU8dTpvfcOJ+N0J/nU11atmuaJOm6JVQ2ZWSnwUIZE8Dtgjrv/KZp+Cxjh7qtr2mdTSwQtWqS/ecws/MJvqC1b4KGHQinhiSfCP8GoUaGUcMwx+oeQxrVpU3gaX7qT/fLlIRlUKS4OVZepw3777XqSb91a39NcyNf7CLoB76dMr4zm7ZIIzGwCMAGgZ8+eOQmusfTsmb5E0Fgfo6goXFU0ejS8/Tb87ndw++1w332hGHveeTBuXPinE6nJl1/CRx/B++9nHlat2nmb9u1h//1h4EA47bQw/pWvhNcuXXSjZFORZIngIeDn7v5cNP0UcIW71/hzv6mVCBqzjSBbmzbBn/8cSgkvvhh+mZ1+ekgKuou56fnXv8Iv7nffDe1CrVqlH1q2zLysqrqkphP9qlWhhJmquBh69Ngx9O6988m+Qwf9mm8q8rVEsArokTLdPZrXrFSd7Ot71VB9FBfDmWeG4dVXQ0K4+264885wSdvhh4dHbA4bFhJDvl3mlq82bgz10C1bNv6+168PVSxLl4aSXer46horS+uvqAi6dQsn+GHDdj7hVw0dO+pEXwiSLBGcCHyfHY3F0919SG37bGolgnyxbl240uj550N/R6+9FtouzODQQ3ckhiOOCL/0Cu2f3z1cqbJiRRiWL9/5dcWKcCc4hMS51151H/bYI/zqrn6iX7o0XFGTar/9wt/hgAN2vPbqFRLR1q27Dtu2pZ9ffZ3OnXec5PfdV6XDQpLUVUN/AkYAnYCPgKuAIgB3vzm6fPR64DjC5aPfra1aCJQIGsu6dfCPf4Sk8MIL8NJLsHZtWNap046kMGxYuE66TZtk482Ge6jn3rYt/fDFF+HmvEwn+/Xrd95f27bh6q7S0vDavXu4Fv3zzzMPVccwG1267DjRp570998/vLdIY9INZVKrL7+EN9/ckRhefBHeeissa9Uq3B4/eHD4RbplS/iFme61pnlbt4b3qTphu+8Ysp1Od4KvOvnX9Sqsvffe+USfOl5aGpbXtWS0bVvoeiBdkqisDJf2VtWx62QvuaREIPXyySehpPDiiyE5LFwYTsatWoX65eqv6ealvrZqFaoiWrQIJ9iqoS7TLVvuOrRokX5+9aGoKJyIq072ahuRQpKvjcWS5zp1Cj2hjhyZdCQiEic1FYmIFDglAhGRAqdEICJS4JQIREQKnBKBiEiBUyJoAmbODJc8tmgRXmfOTDoiEWlOdPlonqvead2KFWEa4u2vSEQKh0oEeW7KlJ17LoUwPWVKMvGISPOjRJDn3nuvbvNFROpKiSDPZXqATRN7Po+I5DElgjw3bdquPX+2aRPmi4g0BiWCPDd2bHiaWUlJ6HStpCTep5uJSOHRVUNNwNixOvGLSHxUIhARKXBKBCIiBU6JQESkwCkRiIgUOCUCEZECp0RQANRpnYjURJePNnPqtE5EaqMSQTOnTutEpDZKBM2cOq0TkdooETRz6rRORGqjRNDMqdM6EamNEkEzp07rRKQ2umqoAKjTOhGpiUoEIiIFTolAsqKb0kSar1gTgZkdZ2ZvmdlSM5ucZvk4M1tjZguj4Zw445H6qbopbcUKcN9xU5qSgUjzEFsiMLOWwA3A8cAhwBgzOyTNqve4e1k03BpXPFJ/uilNpHmLs0QwBFjq7u+4+xfALGBUjO8nMdFNaSLNW5yJoBvwfsr0ymhedd82s0Vm9hcz6xFjPFJPuilNpHlLurH4QaDU3fsBTwB3plvJzCaY2Twzm7dmzZqcBii6KU2kuYszEawCUn/hd4/mbefuFe6+OZq8FRiUbkfuPsPdy929vHPnzrEEK5nppjSR5i3OG8peAQ4ws16EBHAG8G+pK5hZF3dfHU2eBLwZYzzSALopTaT5iq1E4O5bge8DjxFO8Pe6+2Izu9rMTopWu8DMFpvZq8AFwLi44pFk6T4Ekfxl7p50DHVSXl7u8+bNSzoMqYPqD8eB0Mag6iWR3DGz+e5enm5Z0o3FUgB0H4JIflMikNjpPgSR/KZEILHTfQgi+U2JQGKn+xBE8psSgcSuMe5D0FVHIvHRg2kkJxpyH0L1q46qej+t2q+INIxKBJL3dNWRSLyUCCTv6aojkXgpEUje01VHIvFSIpC81xhXHamxWSQzJQLJew296kiP2hSpmfoakmavtDSc/KsrKYHly3MdjUgy1NeQFLTGaGxW1ZI0Z0oE0uw1tLFZVUvS3CkRSLPX0MbmxriPQSUKyWdKBNLsNbSxuaFVSypRSL5TY7FILRra2KzGaskHaiwWaYCGVi2psVrynRKBSC0aWrWUD43VSiRSEyUCkSyMHRuqcb78MrzWpdfTpBurlUikNkoEIjFLurFaiURqo0QgkgMNKVE0tGpJiST57fOeuzepYdCgQS5SSO6+271NG/dwGg1DmzZhfjZKSnbetmooKclue7P025vl5v0b+vmT3r5qHyUl4ZiVlNRt28bY3t0dmOcZzquJn9jrOigRSCFqyImg0BNJ0tvnQyJyVyIQKXiFnEiS3j7pRFSlpkSgNgKRAtCQNoqGNnY39KqphraRJL19Q9tocvGEPiUCEalVU04kSW+fdCLKSqaiQr4OqhoSKTxJN7YmWbWWizYC9TUkIhKzmTPD5bbvvRd+yU+bVrdSVUO3h5r7GlIiEBEpAOp0TkREMoo1EZjZcWb2lpktNbPJaZbvbmb3RMtfNrPSOOMREZFdxZYIzKwlcANwPHAIMMbMDqm22veAz9x9f+B/gGviikdERNKLs0QwBFjq7u+4+xfALGBUtXVGAXdG438BjjUzizEmERGpJs5E0A14P2V6ZTQv7TruvhVYC3SsviMzm2Bm88xs3po1a2IKV0SkMLVKOoBsuPsMYAaAma0xszQP/ssLnYBPkg6iBvkeH+R/jIqvYRRfwzQkvpJMC+JMBKuAHinT3aN56dZZaWatgPZARU07dffOjRlkYzKzeZkuz8oH+R4f5H+Miq9hFF/DxBVfnFVDrwAHmFkvM9sNOAN4oNo6DwBnR+OnAk97U7uxQUSkiYutRODuW83s+8BjQEvg9+6+2MyuJtzq/ABwG3CXmS0FPiUkCxERyaFY2wjc/RHgkWrzrkwZ3wScFmcMOTYj6QBqke/xQf7HqPgaRvE1TCzxNbkuJkREpHGpiwkRkQKnRCAiUuCUCOrIzHqY2TNm9oaZLTazC9OsM8LM1prZwmi4Mt2+YoxxuZm9Fr33Ll21WjA96uNpkZkNzGFsfVKOy0IzW2dmF1VbJ+fHz8x+b2Yfm9nrKfM6mNkTZvZ29Lp3hm3PjtZ528zOTrdOTPH90syWRH/D+81srwzb1vh9iDG+qWa2KuXveEKGbWvskyzG+O5JiW25mS3MsG2sxy/TOSWn379MDyrQkH4AugADo/F2wP8DDqm2zgjgoQRjXA50qmH5CcCjgAGHAy8nFGdL4EOgJOnjBxwFDAReT5n3C2ByND4ZuCbNdh2Ad6LXvaPxvXMU3zeAVtH4Neniy+b7EGN8U4FLs/gOLAN6A7sBr1b/f4orvmrLfwVcmcTxy3ROyeX3TyWCOnL31e6+IBqvBN5k164z8t0o4A8evATsZWZdEojjWGCZuyd+p7i7zyVcwpwqtS+sO4GT02z6TeAJd//U3T8DngCOy0V87v64h65ZAF4i3LSZiAzHLxvZ9EnWYDXFF/Vv9h3gT439vtmo4ZySs++fEkEDRN1mDwBeTrP4CDN71cweNbO+uY0MBx43s/lmNiHN8mz6gcqFM8j8z5fk8auyr7uvjsY/BPZNs06+HMvxhFJeOrV9H+L0/ajq6vcZqjby4fgdCXzk7m9nWJ6z41ftnJKz758SQT2ZWVvgPuAid19XbfECQnVHf+C3wOwch/dVdx9I6AL8fDM7KsfvX6vobvOTgD+nWZz08duFh3J4Xl5rbWZTgK3AzAyrJPV9uAn4ClAGrCZUv+SjMdRcGsjJ8avpnBL390+JoB7MrIjwB5vp7n+tvtzd17n7+mj8EaDIzDrlKj53XxW9fgzcTyh+p8qmH6i4HQ8scPePqi9I+vil+Kiqyix6/TjNOokeSzMbB4wExkYni11k8X2Ihbt/5O7b3P1L4JYM75v08WsFnALck2mdXBy/DOeUnH3/lAjqKKpPvA14091/nWGd/aL1MLMhhONcY2d6jRjfHmbWrmqc0KD4erXVHgDOsuBwYG1KETRXMv4KS/L4VZPaF9bZwP+mWecx4BtmtndU9fGNaF7szOw44HLgJHffkGGdbL4PccWX2u40OsP7ZtMnWZy+Dixx95XpFubi+NVwTsnd9y+ulvDmOgBfJRTRFgELo+EEYCIwMVrn+8BiwhUQLwHDchhf7+h9X41imBLNT43PCE+PWwa8BpTn+BjuQTixt0+Zl+jxIySl1cAWQj3r9wjPxngKeBt4EugQrVsO3Jqy7XhgaTR8N4fxLSXUD1d9D2+O1u0KPFLT9yFH8d0Vfb8WEU5qXarHF02fQLhSZlku44vm31H1vUtZN6fHr4ZzSs6+f+piQkSkwKlqSESkwCkRiIgUOCUCEZECp0QgIlLglAhERAqcEoFIxMy22c49ozZaT5hmVpra86VIPon1UZUiTcxGdy9LOgiRXFOJQKQWUX/0v4j6pP+Hme0fzS81s6ejTtWeMrOe0fx9LTwf4NVoGBbtqqWZ3RL1Of+4mbWO1r8g6ot+kZnNSuhjSgFTIhDZoXW1qqHTU5atdffDgOuB66J5vwXudPd+hA7fpkfzpwN/99Bp3kDCHakABwA3uHtf4HPg29H8ycCAaD8T4/loIpnpzmKRiJmtd/e2aeYvB45x93eizsE+dPeOZvYJoduELdH81e7eyczWAN3dfXPKPkoJ/cYfEE1fARS5+8/M7G/AekIvq7M96nBPJFdUIhDJjmcYr4vNKePb2NFGdyKh76eBwCtRj5giOaNEIJKd01NeX4zGXyD0lgkwFng2Gn8KmARgZi3NrH2mnZpZC6CHuz8DXAG0B3YplYjESb88RHZobTs/wPxv7l51CeneZraI8Kt+TDTvB8DtZnYZsAb4bjT/QmCGmX2P8Mt/EqHny3RaAndHycKA6e7+eSN9HpGsqI1ApBZRG0G5u3+SdCwicVDVkIhIgVOJQESkwKlEICJS4JQIREQKnBKBiEiBUyIQESlwSgQiIgXu/wMNicio3lsP8wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 304 ms (started: 2021-07-23 15:54:31 +08:00)\n"
     ]
    }
   ],
   "source": [
    "loss = history.history[\"loss\"]\n",
    "val_loss = history.history[\"val_loss\"]\n",
    "epochs = range(1, len(loss) + 1)\n",
    "plt.plot(epochs, loss, \"bo\", label=\"Training loss\")\n",
    "plt.plot(epochs, val_loss, \"b\", label=\"Validation loss\")\n",
    "plt.title(\"Training and validation loss\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "suited-black",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAu9klEQVR4nO3deZgU1dn38e/NgOAIggwuIKsKuISwjfiIUSFqRI0S0CiIiUgiiuKWuCUoEhOeJ26RuCaoQY0kIBoJKmoUNfrGKCACKqKCDjoKCCibI8hyv3+cmqFpemZ6ll5m+ve5rr66qrqq+u6annP3OafqlLk7IiKSuxpkOgAREcksJQIRkRynRCAikuOUCEREcpwSgYhIjlMiEBHJcUoEsgsze8bMzq3tdTPJzIrM7PgU7NfN7KBo+k9mdn0y61bjfYaZ2b+qG6dIRUzXEdQPZrYxZjYf2Axsi+YvcPfJ6Y8qe5hZEfBzd3+hlvfrQGd3X1Jb65pZR+BjoJG7b62VQEUq0DDTAUjtcPempdMVFXpm1lCFi2QLfR+zg5qG6jkz62dmxWZ2jZmtACaZ2V5m9pSZrTKzr6LptjHbvGxmP4+mh5vZ/zOzW6N1Pzazk6q5bicze8XMNpjZC2Z2t5k9Uk7cycT4WzP7T7S/f5lZq5jXf2Jmy8xsjZmNqeD4HGFmK8wsL2bZIDNbGE33MbP/mtlaM1tuZneZ2W7l7OtBM/tdzPxV0Tafm9mIuHVPMbO3zGy9mX1qZuNiXn4lel5rZhvN7MjSYxuzfV8zm2Nm66Lnvskemyoe55ZmNin6DF+Z2fSY1waa2fzoMyw1swHR8p2a4cxsXOnf2cw6Rk1kPzOzT4AXo+XTor/Duug7cljM9rub2W3R33Nd9B3b3cyeNrNL4j7PQjMblOizSvmUCHLDfkBLoAMwkvB3nxTNtwe+Ae6qYPsjgPeBVsDNwANmZtVY92/AbKAAGAf8pIL3TCbGs4HzgH2A3YArAczsUODeaP9tovdrSwLu/gbwNfD9uP3+LZreBlwRfZ4jgeOAiyqImyiGAVE8JwCdgfj+ia+BnwItgFOAUWb2o+i1Y6LnFu7e1N3/G7fvlsDTwB3RZ/sD8LSZFcR9hl2OTQKVHee/EpoaD4v2dXsUQx/gYeCq6DMcAxSV8x6JHAscApwYzT9DOE77APOA2KbMW4HeQF/C9/hqYDvwEHBO6Upm1h3Yn3BspCrcXY969iD8Qx4fTfcDvgWaVLB+D+CrmPmXCU1LAMOBJTGv5QMO7FeVdQmFzFYgP+b1R4BHkvxMiWK8Lmb+IuDZaHosMCXmtT2iY3B8Ofv+HfCXaLoZoZDuUM66lwNPxMw7cFA0/SDwu2j6L8DvY9brErtugv1OAG6PpjtG6zaMeX048P+i6Z8As+O2/y8wvLJjU5XjDLQmFLh7JVjvz6XxVvT9i+bHlf6dYz7bARXE0CJapzkhUX0DdE+wXhPgK0K/C4SEcU8q/qfq+0M1gtywyt03lc6YWb6Z/Tmqaq8nNEW0iG0eibOidMLdS6LJplVctw3wZcwygE/LCzjJGFfETJfExNQmdt/u/jWwprz3Ivz6H2xmjYHBwDx3XxbF0SVqLlkRxfG/hNpBZXaKAVgW9/mOMLOXoiaZdcCFSe63dN/L4pYtI/waLlXesdlJJce5HeFv9lWCTdsBS5OMN5GyY2NmeWb2+6h5aT07ahatokeTRO8VfaenAueYWQNgKKEGI1WkRJAb4k8N+yXQFTjC3fdkR1NEec09tWE50NLM8mOWtatg/ZrEuDx239F7FpS3srsvIhSkJ7FzsxCEJqbFhF+dewK/rk4MhBpRrL8BM4B27t4c+FPMfis7le9zQlNOrPbAZ0nEFa+i4/wp4W/WIsF2nwIHlrPPrwm1wVL7JVgn9jOeDQwkNJ81J9QaSmNYDWyq4L0eAoYRmuxKPK4ZTZKjRJCbmhGq22uj9uYbUv2G0S/sucA4M9vNzI4ETk1RjI8BPzSz70UduzdS+Xf9b8BlhIJwWlwc64GNZnYwMCrJGB4FhpvZoVEiio+/GeHX9qaovf3smNdWEZpkDihn3zOBLmZ2tpk1NLOzgEOBp5KMLT6OhMfZ3ZcT2u7viTqVG5lZaaJ4ADjPzI4zswZmtn90fADmA0Oi9QuBM5KIYTOh1pZPqHWVxrCd0Mz2BzNrE9Uejoxqb0QF/3bgNlQbqDYlgtw0Adid8GvrdeDZNL3vMEKH6xpCu/xUQgGQyASqGaO7vwtcTCjclxPakYsr2ezvhA7MF919dczyKwmF9AbgvijmZGJ4JvoMLwJLoudYFwE3mtkGQp/GozHblgDjgf9YOFvpf+L2vQb4IeHX/BpC5+kP4+JO1gQqPs4/AbYQakVfEPpIcPfZhM7o24F1wL/ZUUu5nvAL/ivgN+xcw0rkYUKN7DNgURRHrCuBt4E5wJfATexcdj0MdCP0OUk16IIyyRgzmwosdveU10ik/jKznwIj3f17mY6lrlKNQNLGzA43swOjpoQBhHbh6RkOS+qwqNntImBipmOpy5QIJJ32I5zauJFwDvwod38roxFJnWVmJxL6U1ZSefOTVEBNQyIiOU41AhGRHFfnBp1r1aqVd+zYMdNhiIjUKW+++eZqd9870Wt1LhF07NiRuXPnZjoMEZE6xczir0Yvo6YhEZEcp0QgIpLjlAhERHKcEoGISI5TIhARyXFKBCIiKTZ5MnTsCA0ahOfJkyvbona3r4wSgYikXKYLwkxuP3kyjBwJy5aBe3geOTL5fdR0+6Rk+hZpVX307t3bRSS9HnnEvUMHd7Pw/MgjVds2P989FGPhkZ+f/D7q+vYdOuy8bemjQ4f0bF8KmOvllKsZL9ir+lAikFxUk4K4ptvX9YIw09ubJd7eLD3bl1IiEKnD6vov2kwXhJnePtOJqFRFiUB9BCIpVtP26TFjoKRk52UlJWF5Orb/5JOqLY/XPv5uzZUsr2/bjx8P+fk7L8vPD8vTsX1SyssQ2fpQjUDqkpr+Gnev+79oM12jyfT2pfvIVNNeKdQ0JFJ9NfknrI1qfaabFupDQZjp7bOBEoFINdW0EKyNjr768ItWMk+JQKSaMv1rvJR+0UpNVZQI1Fks9V5NOmtr2lFaWx19w4ZBURFs3x6ehw1L7/ZSvykRSL1W06sya3rGyLBhMHEidOgAZuF54kQVxJJd6tzN6wsLC113KJNkdewYCv94HTqEX8aVKU0ksadf5uerMJe6x8zedPfCRK+pRiD1Wk2bdvSLXnJBnbtnsUhVtG+fuEaQbNMOhEJfBb/UZ6oRSNarSWdvWq7KFKnjlAgkq9W0s1dNOyKVU2exZLWadvaKSKDOYqmzatrZKyKVUyKQrFbT8/hFpHJKBJLV1NkrknpKBJLV1NkrknpKBJJyNb0xi8bJEUktXVAmKRU/REPp6Z+gAl0kW6hGIClV09skikjqKRFISun0T5Hsp0QgKaXTP0WynxKBpJRO/xTJfilNBGY2wMzeN7MlZnZtgtc7mNksM1toZi+bWdtUxiPpp9M/RbJfysYaMrM84APgBKAYmAMMdfdFMetMA55y94fM7PvAee7+k4r2q7GGRESqLlNjDfUBlrj7R+7+LTAFGBi3zqHAi9H0SwleFxGRFEtlItgf+DRmvjhaFmsBMDiaHgQ0M7OC+B2Z2Ugzm2tmc1etWpWSYEVEclWmO4uvBI41s7eAY4HPgG3xK7n7RHcvdPfCvffeO90x5ryaXhksItktlVcWfwa0i5lvGy0r4+6fE9UIzKwpcLq7r01hTFJFujJYpP5LZY1gDtDZzDqZ2W7AEGBG7Apm1srMSmP4FfCXFMYj1aArg0Xqv5QlAnffCowGngPeAx5193fN7EYzOy1arR/wvpl9AOwL6OzyLKMrg0Xqv5QOOufuM4GZccvGxkw/BjyWyhikZtq3T3yrSF0ZLFJ/ZLqzWLKcrgwWqf+UCKRCujJYpP7T/QikUsOGqeAXqc9UIxARyXFKBCIiOU6JQEQkxykRiIjkOCUCEZEcp0QgIpLjlAhygEYPFZGK6DqCek6jh4pIZVQjqOc0eqiIVEaJoJ7T6KEiUhklgnquvFFCNXqoiJRSIqjnNHqoiFRGiaCe0+ihIlIZnTWUAzR6qIhURDUCEZEcp0QgIpLjlAhERHKcEoGISI5TIhARyXFKBCIiOU6JQEQkxykRiIjkOCWCOkD3ExCRVNKVxVlO9xMQkVRTjSDL6X4CIpJqSgRZTvcTEJFUUyLIcrqfgIikmhJBltP9BEQk1ZQIspzuJyAiqaazhuoA3U8gs957D/7wB5g2Lcw3aRIejRvvmE5mvkUL+OlPYe+9M/pxRHahRCCSgDv8+99w663w9NOhID/zTNhrL9i0acdj8+Yd0xs2wOrViV/btAm2b4ff/hbGjoXRo2G33TL9KUUCJQJJi+3bQ2EY39+RbbZsgcceg9tugzffDL/ef/MbGDWq5r/kFy2CX/4yPP785/Aep5wSmvzSYeNG+Pbb6m+flwfNm9dePJI9lAik2rZtC7+AV66EFSvCc3mPL74Iv7KPOAJOPTU8vvOd9BWCldmwAe6/HyZMCKfmdu0aCuuf/AR237123uPQQ+GZZ8LjF78Ix+CEE+D22+Gww2rnPeJt3w7PPQd33RXe171m+2vbFvr02fHo3Rv23LN2Yq2IO6xZE5rXGqrUqnXmNf1mpFlhYaHPnTs302HUScuWwbp1OzdZVGX66693LtxXrw4FTbzGjWG//WDffXd+QCiUSv98HTrsSArHHhu2S7fiYrjjjlDor18PxxwDV14Zfqk3SOGpFFu2wL33wrhx4W9y4YWh5tGqVe3sf+1amDQJ7rkHliwJf4/hw6F16+rvc9MmWLAAZs8O+4SQyA85ZOfk0K1b9Zu91q+HDz4Ij/ff3/n5669DrWT//cPp0x067PwoXZbttc6q+vpreP55ePJJ+PnP4cgjq7cfM3vT3QsTvqZEkBvGj4frrqvaNmY7d3rm58M+++xawMc/9tyz4l/6n38e2t2ffBJeeAG++QaaNoUTTwxJ4eSTU9+hOn9+aJqZMiUksx//ODTZHH54at833po1IRncey80awY33AAXXwyNGlVvfwsXwt13wyOPhCvQjzoq9EcMHly7fRJr1oSE/sYbITHMng2rVoXXGjeGnj13Tg4HHbTjO7FlCxQVhQI+vrBfvnzHe5iFsbW6doUuXaBTp/C+y5bteBQXh5pprFatdk0OpdMFBaFW0axZSCrZqrg4/H88+SS8+GL4Mda8Odx5Z6ilVocSQY577jk46aRQGAwdmvislkRnvDRqlPqmm5KS8EWfMQOeeioUBGbQt++O2sIhh9ROHO7hWNx2W0hAe+wB558Pl10WCpxMevfd0Fz0r3+Fgu+220JCTOZzb9kC06eH5p9XXgl/v2HDQkLp2TPloQPh2C5btiMpzJ4d+lhKh0fZa69QU1i5EpYuha1bd2xbULCjsI99PvDA8Fkqsm1b+GERmxziH998k3jbZs1CUmjefNfnRMtatAjJaJ99av//Yvt2mDdvR+H/1lth+YEH7vg/OPro6v9AgAwmAjMbAPwRyAPud/ffx73eHngIaBGtc627z6xon0oEVfPJJ6Ew2H9/eP317K42l/fPcMAB4R/h+OND+3B1mrU2bQoF0OLF0KYNXHppGLxvr70y+5ljucPMmSEhfPAB/OAHof/g0EMTr79iBdx3H/zpT6Ew7NQJLroIRoyAli3TG3siW7eGDvLSxPDOO6FpKr7QLyhIXQylfQvLloX/ha++Ck1x69aF5rPY5/jp2GQVq3nzXZNW167QuXPV/r9KSmDWrPBdL/0R1KBBaPqp7R9BkKFEYGZ5wAfACUAxMAcY6u6LYtaZCLzl7vea2aHATHfvWNF+lQiSt3lzaPNevDhU4zt3znREVVNcHP5Bnnwy/MNs3lz5Ng0bll/badECzj0XhgzJ7lM3t2wJbfvjxoVO7FGjwnRBQSjYXn89/PqfNi2se+KJofnnpJOyu7mjLnEPBXVscvjyy/BjIrY569NPd96uXbudk0PpdPv24W+zfPmO73Rps2izZjs3i9ZWP1G8TCWCI4Fx7n5iNP8rAHf/v5h1/gx85O43Revf5u59K9qvEkHyLr44FCj/+AcMGpTpaGrm669DDSEvr/ymrcaN69cZJatXhwTwpz+FfpeRI0On4bx5Yf6880INoEuXTEeau0pK4MMPd+3reP/9kDxKNW4caqIffxzmM3GiRKYSwRnAAHf/eTT/E+AIdx8ds05r4F/AXsAewPHu/maCfY0ERgK0b9++97Jly1ISc30yeTKccw5cdRXcfHOmo5GaePdduOKKkAS+852Q4M85J3SwS3ZyD53nsR3iRUXQo0fmTp3O5kTwiyiG26IawQPAd9w9wUmJgWoElXvnnXC+fmFhaFKpT7+Sc5V76Gjdd9/sufZC6paKEkEqB537DGgXM982WhbrZ8CjAO7+X6AJkKIWstywfj2cfnpoOpg6VUmgvjAL1wIoCUgqVJoIzOxUM6tOwpgDdDazTma2GzAEmBG3zifAcdH7HEJIBKuq8V5C+NU4YkTo0Jo6NRQcIiKVSaaAPwv40MxuNrODk92xu28FRgPPAe8Bj7r7u2Z2o5mdFq32S+B8M1sA/B0Y7nXtwoYsMmECPP44/P734WwhEZFkJNVHYGZ7AkOB8wAHJgF/d/cNqQ1vV+ojSOzVV6F/fxg4MAyapiYEEYlV4z4Cd18PPAZMAVoDg4B5ZnZJrUVZj02eHK5cbdAgPE+eXLv7X7ECzjorXHj1l78oCYhI1VTalRg145wHHAQ8DPRx9y/MLB9YBNyZ2hDrtsmTw/nfpZfaL1sW5qF2bjazdWsYNmLt2jB8goYJFpGqSqZGcDpwu7t3c/db3P0LAHcvIZz1IxUYM2ZHEihVUhKW14brroOXXw6jZ3brVjv7FJHckszJheOAsjEBzWx3YF93L3L3WakKrL745JOqLa+Kf/4TbroJLrig+iMSiogkUyOYBsRe4LUtWiZJaN++asuTtWRJGDensDCcLSQiUl3JJIKG7l52g7toOouH7Mou48fvOiJhfn5YXl0lJeGisby8MPBYZUP1iohUJJlEsCrmvH/MbCCwOnUh1S/DhsHEiWGQKbPwPHFi9TuK3cNYM2+/HW4+kulx9EWk7kumj+BCYLKZ3QUY8Cnw05RGVc8MG1Y7ZwgBPPAAPPggjB0bhh0WEampShOBuy8F/sfMmkbzG1MelST05pth3Pkf/CAkAhGR2pDUkGRmdgpwGNDEoquV3P3GFMYlkfXrd9xy75JLwm3yJk/WDUhEpPYkc0HZn4B8oD9wP3AGMDvFceWE0qGFSwv6Tz7Z9X6rsTe32H13eOml1N3BSERyUzI1gr7u/l0zW+juvzGz24BnUh1YffPll+FOU0uX7ij0P/lk19svNm8eOpQ7dAgDx7Vvv2O+S5fsuseuiNQPySSCTdFziZm1AdYQxhuSKhgxIlwAtt9+oVDv2RN+9KMdhXxpga8hIkQk3ZJJBE+aWQvgFmAeYfTR+1IZVH3zxBMhCdx8c7h1pIhINqkwEUQ3pJnl7muBx83sKaCJu6+raDvZYf360MnbvTtcfnmmoxER2VWFF5RF9w6+O2Z+s5JA1Vx/PXz+eRgUrlGjTEcjIrKrZK4snmVmp5tplPuqmjMH7rwTLroo3ExeRCQbJZMILiAMMrfZzNab2QYzW5/iuOq8rVvDfQdat67ZuEIiIqmWzJXFzdIRSH1zxx0wf364baTOBBKRbJbMBWUJb4Pu7q/Ufjj1w7JloW/g1FNh8OBMRyMiUrFkTh+NPeGxCdAHeBP4fkoiquNKRwc1g7vu0v2DRST7JdM0dGrsvJm1AyakKqC67vHH4emn4bbban7zGRGRdEimszheMXBIbQdSH6xbB5deGq4avvTSTEcjIpKcZPoI7iRcTQwhcfQgXGEsccaMCYPIzZgBDZMa11VEJPOSKa7mxkxvBf7u7v9JUTx11uuvwz33hKuICwszHY2ISPKSSQSPAZvcfRuAmeWZWb67l6Q2tLpjyxa44AJo0wZ+97tMRyMiUjVJXVkM7B4zvzvwQmrCqZsmTICFC8NZQs101YWI1DHJJIImsbenjKbzUxdS3fLxx3DDDTBwYBhWWkSkrkkmEXxtZr1KZ8ysN/BN6kKqO9zDOEJ5eWFMIRGRuiiZPoLLgWlm9jlgwH7AWakMqq6YNg2efTY0DbVrl+loRESqJ5kLyuaY2cFA12jR++6+JbVhZb+1a+Gyy6B3bxg9OtPRiIhUX6VNQ2Z2MbCHu7/j7u8ATc3sotSHlt1+9Sv44guYODE0DYmI1FXJ9BGcH92hDAB3/wo4P2UR1QH//W+4Ef1ll0GvXpWvLyKSzZJJBHmxN6Uxszxgt9SFlN22bAn3GWjXDm68MdPRiIjUXDKdxc8CU83sz9H8BcAzqQspu912G7zzThhGomnTTEcjIlJzySSCa4CRwIXR/ELCmUM556OP4De/CfcYOPXUytcXEakLKm0aim5g/wZQRLgXwfeB91IbVnaZPBk6dIADD4TNm6Ffv0xHJCJSe8qtEZhZF2Bo9FgNTAVw9/7pCS07TJ4c+gRKopGV3OHaa6FlSxg2LLOxiYjUhopqBIsJv/5/6O7fc/c7gW3pCSt7jBmzIwmUKikJy0VE6oOKEsFgYDnwkpndZ2bHEa4szimffFK15SIidU25icDdp7v7EOBg4CXCUBP7mNm9ZvaDZHZuZgPM7H0zW2Jm1yZ4/XYzmx89PjCztdX7GKnTtm3i5boNpYjUF8l0Fn/t7n+L7l3cFniLcCZRhaLrDe4GTgIOBYaa2aFx+77C3Xu4ew/gTuAfVf8IqdWnz67L8vNh/Pj0xyIikgpVumexu3/l7hPd/bgkVu8DLHH3j9z9W2AKMLCC9YcCf69KPKm2ahU891y441iHDmAWnidOVEexiNQfqbyz7v7ApzHzxcARiVY0sw5AJ+DFcl4fSbiWgfZpbJMZPz50DD/8MBxySNreVkQkrapUI0ihIcBjpbfDjBfVQgrdvXDvvfdOS0BFRXDvvXDeeUoCIlK/pTIRfAbEjtLfNlqWyBCyrFlo7Fho0ADGjct0JCIiqZXKRDAH6GxmncxsN0JhPyN+peheB3sB/01hLFWycCE88ghcckn5Zw2JiNQXKUsE7r4VGA08RxiS4lF3f9fMbjSz02JWHQJMcXdPVSxV9etfQ/Pm4QpiEZH6LpWdxbj7TGBm3LKxcfPjUhlDVb36Kjz9NPz+92EYCRGR+i5bOouzgjtccw20aROahUREckFKawR1zYwZ4e5jEyeGi8ZERHKBagSRbdtC30DXruGUURGRXKEaQeThh2HRInjsMWiooyIiOUQ1AuCbb8J1A336hLuPiYjkEv32Be6+G4qLQ63Acm6gbRHJdTlfI1i7Fv73f+HEE6F/Tt17TUQkyPlEcPPN8NVX8H//l+lIREQyI6cTweefw4QJMHQo9OyZ6WhERDIjpxPBjTfCli3w299mOhIRkczJ2UTwwQdw//1wwQVw4IGZjkZEJHNyNhFcdx00aQLXX5/pSEREMisnE8HcuTBtGvziF7DvvpmORkQks3IyEVx7LbRqBVdemelIREQyL+cuKHv+eZg1K5wttOeemY5GRCTzcqpGsH17qA107AgXXpjpaEREskNO1QimTYN588JQEo0bZzoaEZHskDM1gm+/hTFjoFs3OPvsTEcjIpI9cqZGcP/9sHQpPPUU5OVlOhoRkeyRMzWCI46AX/4STj4505GIiGSXnKkR9O4dHiIisrOcqRGIiEhiSgQiIjlOiUBEJMcpEYiI5DglAhGRHKdEICKS45QIRERynBKBiEiOUyIQEclxSgQiIjlOiUBEJMcpEYiI5DglAhGRHKdEICKS45QIRERynBKBiEiOUyIQEclxSgQiIjkupbeqNLMBwB+BPOB+d/99gnXOBMYBDixw97NTGZNIfbJlyxaKi4vZtGlTpkORLNGkSRPatm1Lo0aNkt4mZYnAzPKAu4ETgGJgjpnNcPdFMet0Bn4FHOXuX5nZPqmKR6Q+Ki4uplmzZnTs2BEzy3Q4kmHuzpo1ayguLqZTp05Jb5fKpqE+wBJ3/8jdvwWmAAPj1jkfuNvdvwJw9y9SGI9IvbNp0yYKCgqUBAQAM6OgoKDKNcRUJoL9gU9j5oujZbG6AF3M7D9m9nrUlLQLMxtpZnPNbO6qVatSFK5I3aQkILGq833IdGdxQ6Az0A8YCtxnZi3iV3L3ie5e6O6Fe++9d3ojFBGp51KZCD4D2sXMt42WxSoGZrj7Fnf/GPiAkBhEJAUmT4aOHaFBg/A8eXLN9rdmzRp69OhBjx492G+//dh///3L5r/99tsKt507dy6XXnpppe/Rt2/fmgUplUrlWUNzgM5m1omQAIYA8WcETSfUBCaZWStCU9FHKYxJJGdNngwjR0JJSZhftizMAwwbVr19FhQUMH/+fADGjRtH06ZNufLKK8te37p1Kw0bJi5mCgsLKSwsrPQ9XnvtteoFl0Hbtm0jLy8v02EkLWU1AnffCowGngPeAx5193fN7EYzOy1a7TlgjZktAl4CrnL3NamKSSSXjRmzIwmUKikJy2vT8OHDufDCCzniiCO4+uqrmT17NkceeSQ9e/akb9++vP/++wC8/PLL/PCHPwRCEhkxYgT9+vXjgAMO4I477ijbX9OmTcvW79evH2eccQYHH3www4YNw90BmDlzJgcffDC9e/fm0ksvLdtvrKKiIo4++mh69epFr169dkowN910E926daN79+5ce+21ACxZsoTjjz+e7t2706tXL5YuXbpTzACjR4/mwQcfBKBjx45cc8019OrVi2nTpnHfffdx+OGH0717d04//XRKooO/cuVKBg0aRPfu3enevTuvvfYaY8eOZcKECWX7HTNmDH/84x9r+qdIWkqvI3D3mcDMuGVjY6Yd+EX0EJEU+uSTqi2vieLiYl577TXy8vJYv349r776Kg0bNuSFF17g17/+NY8//vgu2yxevJiXXnqJDRs20LVrV0aNGrXLufBvvfUW7777Lm3atOGoo47iP//5D4WFhVxwwQW88sordOrUiaFDhyaMaZ999uH555+nSZMmfPjhhwwdOpS5c+fyzDPP8M9//pM33niD/Px8vvzySwCGDRvGtddey6BBg9i0aRPbt2/n008/TbjvUgUFBcybNw8IzWbnn38+ANdddx0PPPAAl1xyCZdeeinHHnssTzzxBNu2bWPjxo20adOGwYMHc/nll7N9+3amTJnC7Nmzq3zcqyuliUBEskf79qE5KNHy2vbjH/+4rGlk3bp1nHvuuXz44YeYGVu2bEm4zSmnnELjxo1p3Lgx++yzDytXrqRt27Y7rdOnT5+yZT169KCoqIimTZtywAEHlJ03P3ToUCZOnLjL/rds2cLo0aOZP38+eXl5fPDBBwC88MILnHfeeeTn5wPQsmVLNmzYwGeffcagQYOAcJFWMs4666yy6XfeeYfrrruOtWvXsnHjRk488UQAXnzxRR5++GEA8vLyaN68Oc2bN6egoIC33nqLlStX0rNnTwoKCpJ6z9qgRCCSI8aP37mPACA/PyyvbXvssUfZ9PXXX0///v154oknKCoqol+/fgm3ady4cdl0Xl4eW7durdY65bn99tvZd999WbBgAdu3b0+6cI/VsGFDtm/fXjYff75+7OcePnw406dPp3v37jz44IO8/PLLFe775z//OQ8++CArVqxgxIgRVY6tJjJ9+qiIpMmwYTBxInToAGbheeLE6ncUJ2vdunXsv3+4hKi0Pb02de3alY8++oiioiIApk6dWm4crVu3pkGDBvz1r39l27ZtAJxwwglMmjSprA3/yy+/pFmzZrRt25bp06cDsHnzZkpKSujQoQOLFi1i8+bNrF27llmzZpUb14YNG2jdujVbtmxhcszpWccddxz33nsvEDqV161bB8CgQYN49tlnmTNnTlntIV2UCERyyLBhUFQE27eH51QnAYCrr76aX/3qV/Ts2bNKv+CTtfvuu3PPPfcwYMAAevfuTbNmzWjevPku61100UU89NBDdO/encWLF5f9eh8wYACnnXYahYWF9OjRg1tvvRWAv/71r9xxxx1897vfpW/fvqxYsYJ27dpx5pln8p3vfIczzzyTnj17lhvXb3/7W4444giOOuooDj744LLlf/zjH3nppZfo1q0bvXv3ZtGiMOrObrvtRv/+/TnzzDPTfsaRlfa61xWFhYU+d+7cTIchkhXee+89DjnkkEyHkXEbN26kadOmuDsXX3wxnTt35oorrsh0WFWyffv2sjOOOneu2eVUib4XZvamuyc8X1c1AhGp8+677z569OjBYYcdxrp167jgggsyHVKVLFq0iIMOOojjjjuuxkmgOtRZLCJ13hVXXFHnagCxDj30UD76KHPX0qpGICKS45QIRERynBKBiEiOUyIQEclxSgQiUm39+/fnueee22nZhAkTGDVqVLnb9OvXj9JTwE8++WTWrl27yzrjxo0rO5+/PNOnTy87Bx9g7NixvPDCC1WIXkopEYhItQ0dOpQpU6bstGzKlCnlDvwWb+bMmbRo0aJa7x2fCG688UaOP/74au0rU0qvbs40JQKReuLyy6Ffv9p9XH55xe95xhln8PTTT5fdhKaoqIjPP/+co48+mlGjRlFYWMhhhx3GDTfckHD7jh07snr1agDGjx9Ply5d+N73vlc2VDWQcDjn1157jRkzZnDVVVfRo0cPli5dyvDhw3nssccAmDVrFj179qRbt26MGDGCzZs3l73fDTfcQK9evejWrRuLFy/eJaZcHK5aiUBEqq1ly5b06dOHZ555Bgi1gTPPPBMzY/z48cydO5eFCxfy73//m4ULF5a7nzfffJMpU6Ywf/58Zs6cyZw5c8peGzx4MHPmzGHBggUccsghPPDAA/Tt25fTTjuNW265hfnz53PggQeWrb9p0yaGDx/O1KlTefvtt9m6dWvZ2D4ArVq1Yt68eYwaNSph81PpcNXz5s1j6tSpZXdRix2uesGCBVx99dVAGK764osvZsGCBbz22mu0bt260uNWOlz1kCFDEn4+oGy46gULFjBv3jwOO+wwRowYUTZyaelw1eecc06l71cZXVAmUk/E/FBMq9LmoYEDBzJlypSyguzRRx9l4sSJbN26leXLl7No0SK++93vJtzHq6++yqBBg8qGgj7ttNPKXitvOOfyvP/++3Tq1IkuXboAcO6553L33XdzeVS9GTx4MAC9e/fmH//4xy7b5+Jw1TlRI6jt+7SKyA4DBw5k1qxZzJs3j5KSEnr37s3HH3/MrbfeyqxZs1i4cCGnnHLKLkM2J2v48OHcddddvP3229xwww3V3k+p0qGsyxvGOna46rlz51Z67+VEqjpcdVU+X+lw1ZMmTaq14arrfSIovU/rsmXgvuM+rUoGIrWjadOm9O/fnxEjRpR1Eq9fv5499tiD5s2bs3LlyrKmo/Icc8wxTJ8+nW+++YYNGzbw5JNPlr1W3nDOzZo1Y8OGDbvsq2vXrhQVFbFkyRIgjCJ67LHHJv15cnG46nqfCNJ1n1aRXDZ06FAWLFhQlgi6d+9Oz549Ofjggzn77LM56qijKty+V69enHXWWXTv3p2TTjqJww8/vOy18oZzHjJkCLfccgs9e/Zk6dKlZcubNGnCpEmT+PGPf0y3bt1o0KABF154YdKfJReHq673w1A3aBBqAvHMwpjsInWZhqHOPckMV61hqOOUdz/WVNynVUQklVI1XHW9P2sonfdpFRFJpVQNV13vawSZuk+rSLrUteZdSa3qfB/qfY0AQqGvgl/qoyZNmrBmzRoKCgows0yHIxnm7qxZsybp6xlK5UQiEKmv2rZtS3FxMatWrcp0KJIlmjRpQtu2bau0jRKBSB3WqFEjOnXqlOkwpI6r930EIiJSMSUCEZEcp0QgIpLj6tyVxWa2CliW6TjK0QpYnekgKqD4aibb44Psj1Hx1UxN4uvg7nsneqHOJYJsZmZzy7uEOxsovprJ9vgg+2NUfDWTqvjUNCQikuOUCEREcpwSQe2amOkAKqH4aibb44Psj1Hx1UxK4lMfgYhIjlONQEQkxykRiIjkOCWCKjKzdmb2kpktMrN3zeyyBOv0M7N1ZjY/eoxNc4xFZvZ29N673M7NgjvMbImZLTSzXmmMrWvMcZlvZuvN7PK4ddJ+/MzsL2b2hZm9E7OspZk9b2YfRs97lbPtudE6H5rZuWmK7RYzWxz9/Z4wsxblbFvhdyHFMY4zs89i/o4nl7PtADN7P/o+XpvG+KbGxFZkZvPL2Talx7C8MiWt3z9316MKD6A10CuabgZ8ABwat04/4KkMxlgEtKrg9ZOBZwAD/gd4I0Nx5gErCBe6ZPT4AccAvYB3YpbdDFwbTV8L3JRgu5bAR9HzXtH0XmmI7QdAw2j6pkSxJfNdSHGM44Ark/gOLAUOAHYDFsT/P6UqvrjXbwPGZuIYllempPP7pxpBFbn7cnefF01vAN4D9s9sVFU2EHjYg9eBFmbWOgNxHAcsdfeMXynu7q8AX8YtHgg8FE0/BPwowaYnAs+7+5fu/hXwPDAg1bG5+7/cfWs0+zpQtXGHa1k5xy8ZfYAl7v6Ru38LTCEc91pVUXwWbuRwJvD32n7fZFRQpqTt+6dEUANm1hHoCbyR4OUjzWyBmT1jZoelNzIc+JeZvWlmIxO8vj/wacx8MZlJZkMo/58vk8ev1L7uvjyaXgHsm2CdbDiWIwg1vEQq+y6k2uio+eov5TRtZMPxOxpY6e4flvN62o5hXJmStu+fEkE1mVlT4HHgcndfH/fyPEJzR3fgTmB6msP7nrv3Ak4CLjazY9L8/pUys92A04BpCV7O9PHbhYd6eNada21mY4CtwORyVsnkd+Fe4ECgB7Cc0PySjYZScW0gLcewojIl1d8/JYJqMLNGhD/YZHf/R/zr7r7e3TdG0zOBRmbWKl3xuftn0fMXwBOE6nesz4B2MfNto2XpdBIwz91Xxr+Q6eMXY2Vpk1n0/EWCdTJ2LM1sOPBDYFhUUOwiie9Cyrj7Snff5u7bgfvKee+MfhfNrCEwGJha3jrpOIbllClp+/4pEVRR1J74APCeu/+hnHX2i9bDzPoQjvOaNMW3h5k1K50mdCq+E7faDOCnFvwPsC6mCpou5f4Ky+TxizMDKD0L41zgnwnWeQ74gZntFTV9/CBallJmNgC4GjjN3UvKWSeZ70IqY4ztdxpUznvPATqbWaeoljiEcNzT5XhgsbsXJ3oxHcewgjIlfd+/VPWE19cH8D1CFW0hMD96nAxcCFwYrTMaeJdwBsTrQN80xndA9L4LohjGRMtj4zPgbsLZGm8DhWk+hnsQCvbmMcsyevwISWk5sIXQzvozoACYBXwIvAC0jNYtBO6P2XYEsCR6nJem2JYQ2oZLv4N/itZtA8ys6LuQxuP31+j7tZBQqLWOjzGaP5lwpszSVMWYKL5o+YOl37uYddN6DCsoU9L2/dMQEyIiOU5NQyIiOU6JQEQkxykRiIjkOCUCEZEcp0QgIpLjlAhEIma2zXYeGbXWRsI0s46xI1+KZJOGmQ5AJIt84+49Mh2ESLqpRiBSiWg8+pujMelnm9lB0fKOZvZiNKjaLDNrHy3f18I9AhZEj77RrvLM7L5ozPl/mdnu0fqXRmPRLzSzKRn6mJLDlAhEdtg9rmnorJjX1rl7N+AuYEK07E7gIXf/LmHQtzui5XcA//YwaF4vwhWpAJ2Bu939MGAtcHq0/FqgZ7SfC1Pz0UTKpyuLRSJmttHdmyZYXgR8390/igYHW+HuBWa2mjBswpZo+XJ3b2Vmq4C27r45Zh8dCePGd47mrwEaufvvzOxZYCNhlNXpHg24J5IuqhGIJMfLma6KzTHT29jRR3cKYeynXsCcaERMkbRRIhBJzlkxz/+Npl8jjJYJMAx4NZqeBYwCMLM8M2te3k7NrAHQzt1fAq4BmgO71EpEUkm/PER22N12voH5s+5eegrpXma2kPCrfmi07BJgkpldBawCzouWXwZMNLOfEX75jyKMfJlIHvBIlCwMuMPd19bS5xFJivoIRCoR9REUuvvqTMcikgpqGhIRyXGqEYiI5DjVCEREcpwSgYhIjlMiEBHJcUoEIiI5TolARCTH/X/bMwj/IY+iJQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 259 ms (started: 2021-07-23 15:56:28 +08:00)\n"
     ]
    }
   ],
   "source": [
    "plt.clf()\n",
    "acc = history.history[\"accuracy\"]\n",
    "val_acc = history.history[\"val_accuracy\"]\n",
    "plt.plot(epochs, acc, \"bo\", label=\"Training accuracy\")\n",
    "plt.plot(epochs, val_acc, \"b\", label=\"Validation accuracy\")\n",
    "plt.title(\"Training and validation accuracy\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "choice-israeli",
   "metadata": {},
   "source": [
    "该模型在 9 个 epoch 后开始过度拟合。 让我们从头开始训练一个新模型九个时期，然后在测试集上对其进行评估。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "inner-peoples",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4.95 ms (started: 2021-07-23 15:59:13 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(46, activation=\"softmax\")\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "direct-messenger",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 7.55 ms (started: 2021-07-23 15:59:34 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "natural-boundary",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/9\n",
      "18/18 [==============================] - 1s 21ms/step - loss: 2.4781 - accuracy: 0.5412\n",
      "Epoch 2/9\n",
      "18/18 [==============================] - 0s 20ms/step - loss: 1.3457 - accuracy: 0.7116\n",
      "Epoch 3/9\n",
      "18/18 [==============================] - 0s 21ms/step - loss: 1.0071 - accuracy: 0.7901\n",
      "Epoch 4/9\n",
      "18/18 [==============================] - 0s 19ms/step - loss: 0.7916 - accuracy: 0.8324\n",
      "Epoch 5/9\n",
      "18/18 [==============================] - 0s 18ms/step - loss: 0.6206 - accuracy: 0.8706\n",
      "Epoch 6/9\n",
      "18/18 [==============================] - 0s 18ms/step - loss: 0.4950 - accuracy: 0.8971\n",
      "Epoch 7/9\n",
      "18/18 [==============================] - 0s 18ms/step - loss: 0.3934 - accuracy: 0.9186\n",
      "Epoch 8/9\n",
      "18/18 [==============================] - 0s 18ms/step - loss: 0.3183 - accuracy: 0.9300\n",
      "Epoch 9/9\n",
      "18/18 [==============================] - 0s 18ms/step - loss: 0.2678 - accuracy: 0.9390\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f0f27169850>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4.5 s (started: 2021-07-23 15:59:49 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.fit(x_train,\n",
    "          y_train,\n",
    "          epochs=9,\n",
    "          batch_size=512)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "criminal-hobby",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "71/71 [==============================] - 1s 12ms/step - loss: 0.9848 - accuracy: 0.7925\n",
      "time: 1.15 s (started: 2021-07-23 16:00:11 +08:00)\n"
     ]
    }
   ],
   "source": [
    "results = model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "phantom-brief",
   "metadata": {},
   "source": [
    "以下是最终结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "liked-valentine",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.9847751259803772, 0.7925200462341309]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.92 ms (started: 2021-07-23 16:00:46 +08:00)\n"
     ]
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "criminal-tobago",
   "metadata": {},
   "source": [
    "这种方法的准确率达到了~80%。 对于平衡二元分类问题，纯随机分类器达到的准确率将是 50%。 但是在这种情况下，我们有 46 个类，它们的表示可能并不相同。 随机基线的准确度是多少？ 我们可以尝试快速实现一个来凭经验检查这一点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "offshore-protocol",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.35 ms (started: 2021-07-23 16:02:38 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import copy\n",
    "\n",
    "test_labels_copy = copy.copy(test_labels)\n",
    "np.random.shuffle(test_labels_copy)\n",
    "hits_array = np.array(test_labels) == np.array(test_labels_copy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "opened-custody",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.18210151380231523"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.58 ms (started: 2021-07-23 16:02:50 +08:00)\n"
     ]
    }
   ],
   "source": [
    "hits_array.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "growing-perfume",
   "metadata": {},
   "source": [
    "如您所见，随机分类器的分类准确率约为 19%，因此我们模型的结果在这方面看起来相当不错。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hydraulic-change",
   "metadata": {},
   "source": [
    "### 生成对新数据的预测"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "processed-withdrawal",
   "metadata": {},
   "source": [
    "在新样本上调用模型的方法会返回对每个样本的所有 46 个主题进行预测的类概率分布。 让我们为所有测试数据生成主题预测。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "median-recruitment",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 379 ms (started: 2021-07-23 16:04:09 +08:00)\n"
     ]
    }
   ],
   "source": [
    "predictions = model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "printable-wellington",
   "metadata": {},
   "source": [
    "“预测”中的每个条目都是一个长度为 46 的向量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "traditional-cornwall",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(46,)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.52 ms (started: 2021-07-23 16:04:59 +08:00)\n"
     ]
    }
   ],
   "source": [
    "predictions[0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accepting-jacket",
   "metadata": {},
   "source": [
    "这个向量中的系数总和为 1，因为它们形成了一个概率分布："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "comprehensive-fellow",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999998"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.37 ms (started: 2021-07-23 16:06:01 +08:00)\n"
     ]
    }
   ],
   "source": [
    "np.sum(predictions[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "loving-mechanism",
   "metadata": {},
   "source": [
    "最大的条目是预测类别——概率最高的类别："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "cooked-emphasis",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.51 ms (started: 2021-07-23 16:07:03 +08:00)\n"
     ]
    }
   ],
   "source": [
    "np.argmax(predictions[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "further-leader",
   "metadata": {},
   "source": [
    "### 处理标签和损失的不同方式"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "elder-shore",
   "metadata": {},
   "source": [
    "我们之前提到过编码标签的另一种方法是将它们转换为整数张量，像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "classified-friendly",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.04 ms (started: 2021-07-23 16:08:42 +08:00)\n"
     ]
    }
   ],
   "source": [
    "y_train = np.array(train_labels)\n",
    "y_test = np.array(test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "metric-graduation",
   "metadata": {},
   "source": [
    "这种方法唯一会改变的是损失函数的选择。 清单 4.21 中使用的损失函数 `categorical_crossentropy` 期望标签遵循分类编码。 对于整数标签，您应该使用 `sparse_categorical_crossentropy`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "orange-cookbook",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 12.5 ms (started: 2021-07-23 16:10:27 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"sparse_categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "purple-bearing",
   "metadata": {},
   "source": [
    "这个新的损失函数在数学上仍然与 categorical_crossentropy 相同； 它只是有一个不同的界面。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "norman-energy",
   "metadata": {},
   "source": [
    "### 具有足够大的中间层的重要性"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "earned-honor",
   "metadata": {},
   "source": [
    "我们之前提到过，由于最终输出是 46 维的，因此您应该避免使用少于 46 个单元的中间层。 现在让我们看看当您通过具有明显小于 46 维的中间层引入信息瓶颈时会发生什么：例如，4 维。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "oriented-motion",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "63/63 [==============================] - 2s 18ms/step - loss: 2.6865 - accuracy: 0.4049 - val_loss: 1.9452 - val_accuracy: 0.5840\n",
      "Epoch 2/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 1.6122 - accuracy: 0.6438 - val_loss: 1.4965 - val_accuracy: 0.6500\n",
      "Epoch 3/20\n",
      "63/63 [==============================] - 1s 15ms/step - loss: 1.2531 - accuracy: 0.7010 - val_loss: 1.3528 - val_accuracy: 0.6960\n",
      "Epoch 4/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 1.0805 - accuracy: 0.7409 - val_loss: 1.2985 - val_accuracy: 0.7060\n",
      "Epoch 5/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.9716 - accuracy: 0.7578 - val_loss: 1.2775 - val_accuracy: 0.7060\n",
      "Epoch 6/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.8934 - accuracy: 0.7704 - val_loss: 1.2945 - val_accuracy: 0.7010\n",
      "Epoch 7/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.8290 - accuracy: 0.7806 - val_loss: 1.3007 - val_accuracy: 0.7050\n",
      "Epoch 8/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.7728 - accuracy: 0.7915 - val_loss: 1.2949 - val_accuracy: 0.7060\n",
      "Epoch 9/20\n",
      "63/63 [==============================] - 1s 15ms/step - loss: 0.7216 - accuracy: 0.8038 - val_loss: 1.3405 - val_accuracy: 0.7080\n",
      "Epoch 10/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.6788 - accuracy: 0.8098 - val_loss: 1.3808 - val_accuracy: 0.7030\n",
      "Epoch 11/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.6412 - accuracy: 0.8157 - val_loss: 1.3849 - val_accuracy: 0.7050\n",
      "Epoch 12/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.6076 - accuracy: 0.8259 - val_loss: 1.4138 - val_accuracy: 0.7090\n",
      "Epoch 13/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.5764 - accuracy: 0.8322 - val_loss: 1.4703 - val_accuracy: 0.7040\n",
      "Epoch 14/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.5518 - accuracy: 0.8334 - val_loss: 1.4987 - val_accuracy: 0.7000\n",
      "Epoch 15/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.5243 - accuracy: 0.8364 - val_loss: 1.5397 - val_accuracy: 0.7040\n",
      "Epoch 16/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.5066 - accuracy: 0.8400 - val_loss: 1.5756 - val_accuracy: 0.7080\n",
      "Epoch 17/20\n",
      "63/63 [==============================] - 1s 15ms/step - loss: 0.4856 - accuracy: 0.8498 - val_loss: 1.6266 - val_accuracy: 0.7040\n",
      "Epoch 18/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.4646 - accuracy: 0.8599 - val_loss: 1.7012 - val_accuracy: 0.7020\n",
      "Epoch 19/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.4453 - accuracy: 0.8671 - val_loss: 1.7038 - val_accuracy: 0.7090\n",
      "Epoch 20/20\n",
      "63/63 [==============================] - 1s 16ms/step - loss: 0.4276 - accuracy: 0.8737 - val_loss: 1.7647 - val_accuracy: 0.7070\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f0f80717310>"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 21.5 s (started: 2021-07-23 16:13:13 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(4, activation=\"relu\"),\n",
    "    layers.Dense(46, activation=\"softmax\")\n",
    "])\n",
    "\n",
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\"])\n",
    "\n",
    "model.fit(partial_x_train,\n",
    "          partial_y_train,\n",
    "          epochs=20,\n",
    "          batch_size=128,\n",
    "          validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acute-sterling",
   "metadata": {},
   "source": [
    "该模型现在的峰值验证准确度约为 71%，绝对下降了 8%。 这种下降主要是因为您试图将大量信息（足以恢复 46 个类的分离超平面的信息）压缩到太低维的中间空间中。 该模型能够将必要的信息塞入这些四维大多数表示中，但不是全部。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "drawn-tunnel",
   "metadata": {},
   "source": [
    "### 进一步实验"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "harmful-remedy",
   "metadata": {},
   "source": [
    "* 尝试使用更大或更小的层：32 个单元、128 个单元等。\n",
    "* 您在最终的 softmax 分类层之前使用了两个中间层。 现在尝试使用单个中间层或三个中间层。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "upper-issue",
   "metadata": {},
   "source": [
    "### 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "identified-china",
   "metadata": {},
   "source": [
    "以下是你应该从这个例子中学到的东西：\n",
    "* 如果你试图在类中对数据点进行分类，你的模型应该以大小为 N 的 Dense 层结束。\n",
    "* 在单标签、多类分类问题中，您的模型应该以 softmax 激活结束，以便它输出 N 个输出类的概率分布。\n",
    "* 分类交叉熵几乎总是您应该用于此类问题的损失函数。 它最小化了模型输出的概率分布与目标真实分布之间的距离。\n",
    "* 在多类分类中有两种处理标签的方法：\n",
    "    - 通过分类编码（也称为独热编码）对标签进行编码并使用 categorical_crossentropy 作为损失函数\n",
    "    - 将标签编码为整数并使用 sparse_categorical_crossentropy 损失功能\n",
    "* 如果您需要将数据分类为大量类别，则应避免由于中间层太小而在模型中造成信息瓶颈。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "smart-acquisition",
   "metadata": {},
   "source": [
    "## 预测房价：回归示例"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "lucky-router",
   "metadata": {},
   "source": [
    "前两个示例被认为是分类问题，其目标是预测输入数据点的单个离散标签。 另一种常见的机器学习问题是 ，它包括预测一个连续值而不是一个离散标签：例如回归，根据气象数据预测明天的温度； 或根据其规格预测软件项目完成所需的时间。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "technical-bouquet",
   "metadata": {},
   "source": [
    "> 不要混淆和算法回归逻辑回归。 令人困惑的是，逻辑回归不是回归算法——它是一种分类算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "strong-playlist",
   "metadata": {},
   "source": [
    "### 波士顿房价数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "destroyed-adoption",
   "metadata": {},
   "source": [
    "您将尝试预测 1970 年代中期给定波士顿郊区的房屋中位数价格，根据当时郊区的数据点，例如犯罪率、当地财产税率等。 您将使用的数据集与前两个示例有一个有趣的区别。 它的数据点相对较少：只有 506 个，分为 404 个训练样本和 102 个测试样本。 并且每个输入数据（例如，犯罪率）都有不同的尺度。 例如，一些值是比例，取值在 0 到 1 之间； 其他取值介于 1 和 12 之间，其他取值介于 0 和 100 之间，依此类推。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "pressing-chester",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 12.4 ms (started: 2021-07-25 12:37:37 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import boston_housing\n",
    "\n",
    "(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "quiet-garlic",
   "metadata": {},
   "source": [
    "我们来看一下数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "peripheral-encoding",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(404, 13)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 10.1 ms (started: 2021-07-24 15:01:46 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "short-grenada",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(102, 13)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.83 ms (started: 2021-07-24 15:01:48 +08:00)\n"
     ]
    }
   ],
   "source": [
    "test_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "protecting-matthew",
   "metadata": {},
   "source": [
    "如您所见，您有 404 个训练样本和 102 个测试样本，每个样本具有 13 个数值特征，例如人均犯罪率、每户住宅的平均房间数、\n",
    "高速公路等等。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "found-trailer",
   "metadata": {},
   "source": [
    "目标是自住房屋的中值，以千美元为单位："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "backed-russell",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([15.2, 42.3, 50. , 21.1, 17.7, 18.5, 11.3, 15.6, 15.6, 14.4, 12.1,\n",
       "       17.9, 23.1, 19.9, 15.7,  8.8, 50. , 22.5, 24.1, 27.5, 10.9, 30.8,\n",
       "       32.9, 24. , 18.5, 13.3, 22.9, 34.7, 16.6, 17.5, 22.3, 16.1, 14.9,\n",
       "       23.1, 34.9, 25. , 13.9, 13.1, 20.4, 20. , 15.2, 24.7, 22.2, 16.7,\n",
       "       12.7, 15.6, 18.4, 21. , 30.1, 15.1, 18.7,  9.6, 31.5, 24.8, 19.1,\n",
       "       22. , 14.5, 11. , 32. , 29.4, 20.3, 24.4, 14.6, 19.5, 14.1, 14.3,\n",
       "       15.6, 10.5,  6.3, 19.3, 19.3, 13.4, 36.4, 17.8, 13.5, 16.5,  8.3,\n",
       "       14.3, 16. , 13.4, 28.6, 43.5, 20.2, 22. , 23. , 20.7, 12.5, 48.5,\n",
       "       14.6, 13.4, 23.7, 50. , 21.7, 39.8, 38.7, 22.2, 34.9, 22.5, 31.1,\n",
       "       28.7, 46. , 41.7, 21. , 26.6, 15. , 24.4, 13.3, 21.2, 11.7, 21.7,\n",
       "       19.4, 50. , 22.8, 19.7, 24.7, 36.2, 14.2, 18.9, 18.3, 20.6, 24.6,\n",
       "       18.2,  8.7, 44. , 10.4, 13.2, 21.2, 37. , 30.7, 22.9, 20. , 19.3,\n",
       "       31.7, 32. , 23.1, 18.8, 10.9, 50. , 19.6,  5. , 14.4, 19.8, 13.8,\n",
       "       19.6, 23.9, 24.5, 25. , 19.9, 17.2, 24.6, 13.5, 26.6, 21.4, 11.9,\n",
       "       22.6, 19.6,  8.5, 23.7, 23.1, 22.4, 20.5, 23.6, 18.4, 35.2, 23.1,\n",
       "       27.9, 20.6, 23.7, 28. , 13.6, 27.1, 23.6, 20.6, 18.2, 21.7, 17.1,\n",
       "        8.4, 25.3, 13.8, 22.2, 18.4, 20.7, 31.6, 30.5, 20.3,  8.8, 19.2,\n",
       "       19.4, 23.1, 23. , 14.8, 48.8, 22.6, 33.4, 21.1, 13.6, 32.2, 13.1,\n",
       "       23.4, 18.9, 23.9, 11.8, 23.3, 22.8, 19.6, 16.7, 13.4, 22.2, 20.4,\n",
       "       21.8, 26.4, 14.9, 24.1, 23.8, 12.3, 29.1, 21. , 19.5, 23.3, 23.8,\n",
       "       17.8, 11.5, 21.7, 19.9, 25. , 33.4, 28.5, 21.4, 24.3, 27.5, 33.1,\n",
       "       16.2, 23.3, 48.3, 22.9, 22.8, 13.1, 12.7, 22.6, 15. , 15.3, 10.5,\n",
       "       24. , 18.5, 21.7, 19.5, 33.2, 23.2,  5. , 19.1, 12.7, 22.3, 10.2,\n",
       "       13.9, 16.3, 17. , 20.1, 29.9, 17.2, 37.3, 45.4, 17.8, 23.2, 29. ,\n",
       "       22. , 18. , 17.4, 34.6, 20.1, 25. , 15.6, 24.8, 28.2, 21.2, 21.4,\n",
       "       23.8, 31. , 26.2, 17.4, 37.9, 17.5, 20. ,  8.3, 23.9,  8.4, 13.8,\n",
       "        7.2, 11.7, 17.1, 21.6, 50. , 16.1, 20.4, 20.6, 21.4, 20.6, 36.5,\n",
       "        8.5, 24.8, 10.8, 21.9, 17.3, 18.9, 36.2, 14.9, 18.2, 33.3, 21.8,\n",
       "       19.7, 31.6, 24.8, 19.4, 22.8,  7.5, 44.8, 16.8, 18.7, 50. , 50. ,\n",
       "       19.5, 20.1, 50. , 17.2, 20.8, 19.3, 41.3, 20.4, 20.5, 13.8, 16.5,\n",
       "       23.9, 20.6, 31.5, 23.3, 16.8, 14. , 33.8, 36.1, 12.8, 18.3, 18.7,\n",
       "       19.1, 29. , 30.1, 50. , 50. , 22. , 11.9, 37.6, 50. , 22.7, 20.8,\n",
       "       23.5, 27.9, 50. , 19.3, 23.9, 22.6, 15.2, 21.7, 19.2, 43.8, 20.3,\n",
       "       33.2, 19.9, 22.5, 32.7, 22. , 17.1, 19. , 15. , 16.1, 25.1, 23.7,\n",
       "       28.7, 37.2, 22.6, 16.4, 25. , 29.8, 22.1, 17.4, 18.1, 30.3, 17.5,\n",
       "       24.7, 12.6, 26.5, 28.7, 13.3, 10.4, 24.4, 23. , 20. , 17.8,  7. ,\n",
       "       11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 5.37 ms (started: 2021-07-24 15:01:50 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_targets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "stuck-arkansas",
   "metadata": {},
   "source": [
    "价格通常在 10,000 美元到 50,000 美元之间。 如果这听起来很便宜，请记住这是 1970 年代中期，这些价格没有根据通货膨胀进行调整。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "opponent-pasta",
   "metadata": {},
   "source": [
    "### 准备数据"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "facial-champion",
   "metadata": {},
   "source": [
    "将所有范围都大不相同的神经网络值输入到神经网络中将是有问题的。 模型或许能够自动适应这种异构数据，但这肯定会让学习变得更加困难。 处理此类数据的一种普遍最佳做法是进行特征归一化：对于输入数据中的每个特征（输入数据矩阵中的一列），您减去特征的均值并除以标准差，以便 该特征以 0 为中心并具有单位标准偏差。 这在 NumPy 中很容易完成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "abroad-boutique",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.68 ms (started: 2021-07-24 15:01:53 +08:00)\n"
     ]
    }
   ],
   "source": [
    "mean = train_data.mean(axis=0)\n",
    "train_data -= mean\n",
    "std = train_data.std(axis=0)\n",
    "train_data /= std"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "likely-parade",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 427 µs (started: 2021-07-24 15:01:54 +08:00)\n"
     ]
    }
   ],
   "source": [
    "test_data -= mean\n",
    "test_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "senior-tribute",
   "metadata": {},
   "source": [
    "请注意，用于归一化测试数据的数量是使用训练数据计算的。 您永远不应该在工作流程中使用根据测试数据计算的任何数量，即使对于\n",
    "像数据规范化这样简单的事情。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "promising-switzerland",
   "metadata": {},
   "source": [
    "### 构建模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "informed-ensemble",
   "metadata": {},
   "source": [
    "由于可用样本太少，您将使用一个非常小的模型，其中包含两个中间层，每个层有 64 个单元。 一般来说，你拥有的训练数据越少，过拟合就越严重，使用小模型是减轻过拟合的一种方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "handed-tobacco",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 740 µs (started: 2021-07-24 15:01:57 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 因为您需要多次实例化同一个模型，所以您使用一个函数来构造它。\n",
    "\n",
    "def build_model():\n",
    "    model = keras.Sequential([\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(1)\n",
    " ])\n",
    "    model.compile(optimizer=\"rmsprop\", loss=\"mse\", metrics=[\"mae\"])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "developmental-delicious",
   "metadata": {},
   "source": [
    "模型以单个单元结束，没有激活（它将是一个线性层）。 这是标量回归（尝试预测单个连续值的回归）的典型设置。 应用激活函数会限制输出的范围； 例如，如果您将激活函数应用于最后一层，则模型只能学习 sigmoid 来预测 0 到 1 之间的值。这里，因为最后一层是纯线性的，模型可以自由地学习预测任何范围内的值 ."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "modern-theater",
   "metadata": {},
   "source": [
    "请注意，您使用损失函数编译模型 - mse 均方误差，预测与目标之间差异的平方。 这是一种广泛用于回归问题的损失函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proprietary-grace",
   "metadata": {},
   "source": [
    "您还在训练期间监控一个新指标：平均绝对误差 (MAE)。 它是预测和目标之间差异的绝对值。 例如，此问题的 MAE 为 0.5 意味着您的预测平均偏离 500 美元。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unnecessary-twins",
   "metadata": {},
   "source": [
    "### 使用 K 折验证来验证您的方法"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "logical-plastic",
   "metadata": {},
   "source": [
    "要在不断调整模型参数（例如用于训练的 epoch 数）的同时评估模型，您可以将数据拆分为训练集和验证集，就像在前面的示例中所做的那样。 但是因为数据点太少，验证集最终会非常小（例如，大约 100 个示例）。 因此，验证分数可能会发生很大变化，具体取决于您选择用于验证的数据点以及您选择用于训练的数据点：验证分数可能相对于方差验证拆分而言较高。 这将阻止您可靠地评估您的模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vital-edmonton",
   "metadata": {},
   "source": [
    "这种情况下的最佳实践是使用交叉验证（见图 4.8）。 它包括将可用数据分成 K 个分区（通常 K = 4 或 5）的 K 折，实例化相同的 K\n",
    "模型，并在 K-1 个分区上训练每个模型，同时评估剩余的分区。 所用模型的验证分数是获得的 K 个验证分数的平均值。 就代码而言，这很简单。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "qualified-brooks",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/008i3skNgy1gss1f3ryicj31cy0myq57.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "distant-exclusion",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 781 µs (started: 2021-07-24 15:17:35 +08:00)\n"
     ]
    }
   ],
   "source": [
    "k = 4\n",
    "num_val_samples = len(train_data) // k\n",
    "num_epochs = 100\n",
    "all_scores = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "aquatic-elements",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing fold #0\n",
      "Processing fold #1\n",
      "Processing fold #2\n",
      "Processing fold #3\n",
      "time: 1min 4s (started: 2021-07-24 15:17:37 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "for i in range(k):\n",
    "    print(f\"Processing fold #{i}\")\n",
    "    \n",
    "    # 准备验证数据：来自分区 #k 的数据\n",
    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples] \n",
    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "   \n",
    "    # 准备训练数据：来自所有其他分区的数据\n",
    "    partial_train_data = np.concatenate([train_data[:i * num_val_samples],\n",
    "                                         train_data[(i + 1) * num_val_samples:]],\n",
    "                                        axis=0)\n",
    "    partial_train_targets = np.concatenate(\n",
    "        [train_targets[:i * num_val_samples],\n",
    "         train_targets[(i + 1) * num_val_samples:]],\n",
    "        axis=0)\n",
    "    \n",
    "    # 构建 Keras 模型（已编译）\n",
    "    model = build_model()\n",
    "    \n",
    "    # 训练模型（在静默模式下，verbose = 0）\n",
    "    model.fit(partial_train_data, partial_train_targets,\n",
    "              epochs=num_epochs, batch_size=16, verbose=0)\n",
    "    \n",
    "    # 根据验证数据评估模型\n",
    "    val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0) \n",
    "    \n",
    "    #\n",
    "    all_scores.append(val_mae)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "democratic-terrace",
   "metadata": {},
   "source": [
    "使用 num_epochs = 100 运行它会产生以下结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "featured-spyware",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2.1525533199310303,\n",
       " 2.4750516414642334,\n",
       " 2.5435984134674072,\n",
       " 2.3341240882873535]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.23 ms (started: 2021-07-24 15:21:11 +08:00)\n"
     ]
    }
   ],
   "source": [
    "all_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "alternative-monroe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.376331865787506"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.04 ms (started: 2021-07-24 15:21:13 +08:00)\n"
     ]
    }
   ],
   "source": [
    "np.mean(all_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "material-bacteria",
   "metadata": {},
   "source": [
    "不同的运行确实显示了相当不同的验证分数，从 2.1 到 3.1。 平均值 (2.6) 是比任何单个分数都更可靠的指标——这是 K 折交叉验证的全部要点。 在这种情况下，您平均减少 2,600 美元，考虑到价格从 10,000 美元到 50,000 美元不等，这一点很重要。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "spare-firewall",
   "metadata": {},
   "source": [
    "让我们尝试将模型训练更长时间：500 个 epoch。 为了记录模型在每个时期的表现，您将修改训练循环以保存每个时期的验证分数日志。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "pharmaceutical-motivation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing fold #0\n",
      "Processing fold #1\n",
      "Processing fold #2\n",
      "Processing fold #3\n",
      "time: 7min 22s (started: 2021-07-24 15:29:05 +08:00)\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 500\n",
    "all_mae_histories = []\n",
    "\n",
    "for i in range(k):\n",
    "    print(f\"Processing fold #{i}\")\n",
    "    \n",
    "    # 准备验证数据：来自分区 #k 的数据\n",
    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples] \n",
    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "   \n",
    "    # 准备训练数据：来自所有其他分区的数据\n",
    "    partial_train_data = np.concatenate([train_data[:i * num_val_samples],\n",
    "                                         train_data[(i + 1) * num_val_samples:]],\n",
    "                                        axis=0)\n",
    "    partial_train_targets = np.concatenate(\n",
    "        [train_targets[:i * num_val_samples],\n",
    "         train_targets[(i + 1) * num_val_samples:]],\n",
    "        axis=0)\n",
    "    \n",
    "    # 构建 Keras 模型（已编译）\n",
    "    model = build_model()\n",
    "    \n",
    "    # 训练模型（在静默模式下，verbose = 0）\n",
    "    history = model.fit(partial_train_data, partial_train_targets,\n",
    "                        validation_data=(val_data, val_targets),\n",
    "                        epochs=num_epochs, batch_size=16, verbose=0)\n",
    "    \n",
    "    mae_history = history.history[\"val_mae\"]\n",
    "    \n",
    "    all_mae_histories.append(mae_history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "final-heaven",
   "metadata": {},
   "source": [
    "然后，您可以计算所有折叠的每个周期 MAE 分数的平均值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "intelligent-controversy",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 7.96 ms (started: 2021-07-24 15:36:27 +08:00)\n"
     ]
    }
   ],
   "source": [
    "average_mae_history = [np.mean([x[i] for x in all_mae_histories]) for i in range(num_epochs)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "informal-primary",
   "metadata": {},
   "source": [
    "让我们绘制这个图； 见图 4.9。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "acting-anthony",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg height=\"263.163498pt\" version=\"1.1\" viewBox=\"0 0 392.14375 263.163498\" width=\"392.14375pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2021-07-24T15:39:39.494619</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.4.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 263.163498 \n",
       "L 392.14375 263.163498 \n",
       "L 392.14375 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 50.14375 225.607248 \n",
       "L 384.94375 225.607248 \n",
       "L 384.94375 8.167248 \n",
       "L 50.14375 8.167248 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"maa8b0c8047\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.751985\" xlink:href=\"#maa8b0c8047\" y=\"225.607248\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(61.570735 240.205685)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" id=\"DejaVuSans-30\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"125.746701\" xlink:href=\"#maa8b0c8047\" y=\"225.607248\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 100 -->\n",
       "      <g transform=\"translate(116.202951 240.205685)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-31\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"186.741418\" xlink:href=\"#maa8b0c8047\" y=\"225.607248\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 200 -->\n",
       "      <g transform=\"translate(177.197668 240.205685)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-32\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"247.736135\" xlink:href=\"#maa8b0c8047\" y=\"225.607248\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 300 -->\n",
       "      <g transform=\"translate(238.192385 240.205685)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" id=\"DejaVuSans-33\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"308.730851\" xlink:href=\"#maa8b0c8047\" y=\"225.607248\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 400 -->\n",
       "      <g transform=\"translate(299.187101 240.205685)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-34\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"369.725568\" xlink:href=\"#maa8b0c8047\" y=\"225.607248\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 500 -->\n",
       "      <g transform=\"translate(360.181818 240.205685)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-35\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- Epochs -->\n",
       "     <g transform=\"translate(199.628125 253.88381)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 628 4666 \n",
       "L 3578 4666 \n",
       "L 3578 4134 \n",
       "L 1259 4134 \n",
       "L 1259 2753 \n",
       "L 3481 2753 \n",
       "L 3481 2222 \n",
       "L 1259 2222 \n",
       "L 1259 531 \n",
       "L 3634 531 \n",
       "L 3634 0 \n",
       "L 628 0 \n",
       "L 628 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-45\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" id=\"DejaVuSans-70\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" id=\"DejaVuSans-6f\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" id=\"DejaVuSans-63\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" id=\"DejaVuSans-68\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" id=\"DejaVuSans-73\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-45\"/>\n",
       "      <use x=\"63.183594\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"126.660156\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"187.841797\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"242.822266\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"306.201172\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m30573b7ae7\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"212.80413\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 2.5 -->\n",
       "      <g transform=\"translate(27.240625 216.603348)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" id=\"DejaVuSans-2e\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"183.974857\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 5.0 -->\n",
       "      <g transform=\"translate(27.240625 187.774075)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"155.145584\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 7.5 -->\n",
       "      <g transform=\"translate(27.240625 158.944802)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 525 4666 \n",
       "L 3525 4666 \n",
       "L 3525 4397 \n",
       "L 1831 0 \n",
       "L 1172 0 \n",
       "L 2766 4134 \n",
       "L 525 4134 \n",
       "L 525 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-37\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-37\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"126.316311\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 10.0 -->\n",
       "      <g transform=\"translate(20.878125 130.115529)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"97.487038\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 12.5 -->\n",
       "      <g transform=\"translate(20.878125 101.286256)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"68.657765\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 15.0 -->\n",
       "      <g transform=\"translate(20.878125 72.456983)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"39.828492\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 17.5 -->\n",
       "      <g transform=\"translate(20.878125 43.62771)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-37\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m30573b7ae7\" y=\"10.999219\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_15\">\n",
       "      <!-- 20.0 -->\n",
       "      <g transform=\"translate(20.878125 14.798437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_16\">\n",
       "     <!-- Validation MAE -->\n",
       "     <g transform=\"translate(14.798438 154.060685)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 1831 0 \n",
       "L 50 4666 \n",
       "L 709 4666 \n",
       "L 2188 738 \n",
       "L 3669 4666 \n",
       "L 4325 4666 \n",
       "L 2547 0 \n",
       "L 1831 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-56\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" id=\"DejaVuSans-61\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-6c\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2906 2969 \n",
       "L 2906 4863 \n",
       "L 3481 4863 \n",
       "L 3481 0 \n",
       "L 2906 0 \n",
       "L 2906 525 \n",
       "Q 2725 213 2448 61 \n",
       "Q 2172 -91 1784 -91 \n",
       "Q 1150 -91 751 415 \n",
       "Q 353 922 353 1747 \n",
       "Q 353 2572 751 3078 \n",
       "Q 1150 3584 1784 3584 \n",
       "Q 2172 3584 2448 3432 \n",
       "Q 2725 3281 2906 2969 \n",
       "z\n",
       "M 947 1747 \n",
       "Q 947 1113 1208 752 \n",
       "Q 1469 391 1925 391 \n",
       "Q 2381 391 2643 752 \n",
       "Q 2906 1113 2906 1747 \n",
       "Q 2906 2381 2643 2742 \n",
       "Q 2381 3103 1925 3103 \n",
       "Q 1469 3103 1208 2742 \n",
       "Q 947 2381 947 1747 \n",
       "z\n",
       "\" id=\"DejaVuSans-64\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" id=\"DejaVuSans-74\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" id=\"DejaVuSans-6e\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 628 4666 \n",
       "L 1569 4666 \n",
       "L 2759 1491 \n",
       "L 3956 4666 \n",
       "L 4897 4666 \n",
       "L 4897 0 \n",
       "L 4281 0 \n",
       "L 4281 4097 \n",
       "L 3078 897 \n",
       "L 2444 897 \n",
       "L 1241 4097 \n",
       "L 1241 0 \n",
       "L 628 0 \n",
       "L 628 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-4d\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2188 4044 \n",
       "L 1331 1722 \n",
       "L 3047 1722 \n",
       "L 2188 4044 \n",
       "z\n",
       "M 1831 4666 \n",
       "L 2547 4666 \n",
       "L 4325 0 \n",
       "L 3669 0 \n",
       "L 3244 1197 \n",
       "L 1141 1197 \n",
       "L 716 0 \n",
       "L 50 0 \n",
       "L 1831 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-41\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-56\"/>\n",
       "      <use x=\"60.658203\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"121.9375\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"149.720703\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"177.503906\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"240.980469\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"302.259766\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"341.46875\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"369.251953\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"430.433594\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"493.8125\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"525.599609\" xlink:href=\"#DejaVuSans-4d\"/>\n",
       "      <use x=\"611.878906\" xlink:href=\"#DejaVuSans-41\"/>\n",
       "      <use x=\"680.287109\" xlink:href=\"#DejaVuSans-45\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_15\">\n",
       "    <path clip-path=\"url(#pe4218c9b4f)\" d=\"M 65.361932 18.050884 \n",
       "L 65.971879 57.817625 \n",
       "L 67.191773 145.331433 \n",
       "L 67.80172 169.022819 \n",
       "L 68.411668 181.645952 \n",
       "L 69.021615 189.693902 \n",
       "L 69.631562 194.901166 \n",
       "L 70.241509 198.375162 \n",
       "L 71.461403 202.51111 \n",
       "L 72.071351 204.139299 \n",
       "L 73.901192 207.05649 \n",
       "L 74.511139 207.256169 \n",
       "L 75.121086 208.432817 \n",
       "L 75.731034 208.815347 \n",
       "L 76.950928 210.282276 \n",
       "L 77.560875 210.636262 \n",
       "L 78.170822 211.165391 \n",
       "L 78.780769 211.188851 \n",
       "L 79.390717 211.828322 \n",
       "L 80.000664 211.587329 \n",
       "L 80.610611 212.019396 \n",
       "L 81.220558 211.376052 \n",
       "L 81.830505 212.82292 \n",
       "L 83.0504 211.928086 \n",
       "L 83.660347 212.954523 \n",
       "L 84.270294 212.545705 \n",
       "L 85.490188 213.197039 \n",
       "L 86.100135 212.271264 \n",
       "L 86.710083 212.95482 \n",
       "L 87.32003 212.867752 \n",
       "L 87.929977 212.910079 \n",
       "L 88.539924 212.619416 \n",
       "L 89.149871 212.477101 \n",
       "L 89.759819 213.489836 \n",
       "L 90.369766 213.770844 \n",
       "L 90.979713 213.1724 \n",
       "L 92.809554 213.79839 \n",
       "L 93.419502 213.968613 \n",
       "L 94.639396 213.755035 \n",
       "L 95.249343 213.798485 \n",
       "L 96.469237 213.21002 \n",
       "L 97.079185 213.730297 \n",
       "L 97.689132 214.403552 \n",
       "L 98.299079 213.805276 \n",
       "L 98.909026 214.485288 \n",
       "L 99.518973 214.451056 \n",
       "L 100.12892 213.495215 \n",
       "L 100.738868 214.176413 \n",
       "L 101.348815 213.204589 \n",
       "L 101.958762 214.211377 \n",
       "L 102.568709 214.421663 \n",
       "L 103.178656 214.19999 \n",
       "L 103.788603 214.376944 \n",
       "L 104.398551 214.306724 \n",
       "L 105.008498 214.603789 \n",
       "L 105.618445 214.59699 \n",
       "L 106.228392 213.653195 \n",
       "L 106.838339 214.591613 \n",
       "L 107.448286 214.83491 \n",
       "L 108.668181 214.430434 \n",
       "L 109.278128 214.557745 \n",
       "L 109.888075 214.869228 \n",
       "L 110.498022 214.630961 \n",
       "L 111.107969 214.555288 \n",
       "L 111.717917 215.102597 \n",
       "L 112.327864 214.730025 \n",
       "L 112.937811 214.986278 \n",
       "L 114.157705 214.962343 \n",
       "L 114.767652 215.097797 \n",
       "L 115.3776 213.958259 \n",
       "L 115.987547 214.695238 \n",
       "L 116.597494 214.600114 \n",
       "L 117.207441 215.045873 \n",
       "L 117.817388 214.195463 \n",
       "L 118.427335 214.813392 \n",
       "L 119.037283 214.631187 \n",
       "L 119.64723 214.714129 \n",
       "L 120.257177 214.435995 \n",
       "L 120.867124 214.591802 \n",
       "L 121.477071 214.95277 \n",
       "L 122.087018 214.722955 \n",
       "L 122.696966 215.021402 \n",
       "L 123.306913 214.528609 \n",
       "L 123.91686 214.871962 \n",
       "L 124.526807 214.568546 \n",
       "L 125.746701 214.393213 \n",
       "L 126.356649 214.660803 \n",
       "L 126.966596 215.146992 \n",
       "L 127.576543 215.181937 \n",
       "L 128.796437 214.183545 \n",
       "L 129.406384 215.349246 \n",
       "L 130.016332 215.072216 \n",
       "L 130.626279 214.951203 \n",
       "L 131.236226 215.048777 \n",
       "L 131.846173 214.84922 \n",
       "L 132.45612 215.006102 \n",
       "L 133.066067 214.252737 \n",
       "L 133.676015 214.73291 \n",
       "L 134.285962 213.529335 \n",
       "L 135.505856 214.477688 \n",
       "L 136.115803 214.285885 \n",
       "L 136.72575 215.432001 \n",
       "L 137.945645 215.261558 \n",
       "L 138.555592 215.511603 \n",
       "L 139.165539 215.119868 \n",
       "L 139.775486 214.514892 \n",
       "L 140.385433 215.663344 \n",
       "L 140.995381 215.29957 \n",
       "L 141.605328 215.613554 \n",
       "L 142.215275 215.137856 \n",
       "L 142.825222 215.539603 \n",
       "L 143.435169 215.278486 \n",
       "L 144.045116 214.390925 \n",
       "L 144.655064 214.204298 \n",
       "L 145.265011 215.148757 \n",
       "L 145.874958 215.103931 \n",
       "L 146.484905 214.848343 \n",
       "L 147.094852 215.210884 \n",
       "L 147.704799 214.586752 \n",
       "L 148.314747 214.815027 \n",
       "L 148.924694 214.623099 \n",
       "L 149.534641 215.486987 \n",
       "L 150.144588 215.168505 \n",
       "L 150.754535 214.563311 \n",
       "L 151.364482 214.513439 \n",
       "L 151.97443 214.705439 \n",
       "L 152.584377 215.723611 \n",
       "L 153.804271 215.121807 \n",
       "L 154.414218 214.542571 \n",
       "L 155.024165 215.523282 \n",
       "L 155.634113 214.62309 \n",
       "L 156.24406 215.14349 \n",
       "L 156.854007 215.273757 \n",
       "L 157.463954 215.251723 \n",
       "L 158.683848 214.720148 \n",
       "L 159.293796 214.28207 \n",
       "L 159.903743 214.971529 \n",
       "L 160.51369 215.065168 \n",
       "L 161.123637 214.709108 \n",
       "L 161.733584 215.389451 \n",
       "L 162.343531 215.283759 \n",
       "L 162.953479 215.052893 \n",
       "L 163.563426 214.214193 \n",
       "L 164.173373 215.476263 \n",
       "L 164.78332 215.272982 \n",
       "L 165.393267 214.866335 \n",
       "L 166.003214 215.171908 \n",
       "L 166.613162 214.691681 \n",
       "L 167.223109 214.586151 \n",
       "L 167.833056 214.189193 \n",
       "L 168.443003 214.425013 \n",
       "L 169.05295 214.505219 \n",
       "L 169.662897 214.91241 \n",
       "L 170.272845 214.845544 \n",
       "L 170.882792 214.607306 \n",
       "L 171.492739 215.072364 \n",
       "L 172.102686 215.071997 \n",
       "L 172.712633 214.349221 \n",
       "L 173.32258 214.87933 \n",
       "L 173.932528 215.119727 \n",
       "L 174.542475 214.562895 \n",
       "L 175.152422 213.849988 \n",
       "L 175.762369 213.903257 \n",
       "L 176.372316 214.876356 \n",
       "L 176.982263 214.177979 \n",
       "L 177.592211 214.848061 \n",
       "L 178.202158 214.283726 \n",
       "L 178.812105 215.01305 \n",
       "L 179.422052 213.159565 \n",
       "L 180.031999 215.092994 \n",
       "L 180.641946 214.879365 \n",
       "L 181.251894 214.303147 \n",
       "L 181.861841 214.539741 \n",
       "L 182.471788 214.426004 \n",
       "L 183.081735 214.641738 \n",
       "L 183.691682 213.907264 \n",
       "L 184.301629 214.589369 \n",
       "L 184.911577 214.13609 \n",
       "L 185.521524 214.062747 \n",
       "L 186.131471 214.82706 \n",
       "L 186.741418 213.282375 \n",
       "L 187.351365 214.285411 \n",
       "L 187.961312 213.980251 \n",
       "L 188.57126 214.373627 \n",
       "L 189.181207 214.117982 \n",
       "L 189.791154 214.257542 \n",
       "L 190.401101 214.176948 \n",
       "L 191.011048 214.67326 \n",
       "L 191.620995 214.136306 \n",
       "L 192.230943 215.037687 \n",
       "L 192.84089 214.145001 \n",
       "L 193.450837 214.333137 \n",
       "L 194.060784 214.098111 \n",
       "L 194.670731 214.216166 \n",
       "L 195.280678 214.497232 \n",
       "L 195.890626 214.245641 \n",
       "L 197.11052 213.425855 \n",
       "L 198.330414 214.020693 \n",
       "L 199.550309 214.293554 \n",
       "L 200.160256 213.666128 \n",
       "L 200.770203 213.689331 \n",
       "L 201.38015 213.58292 \n",
       "L 201.990097 214.415614 \n",
       "L 202.600044 213.521393 \n",
       "L 203.209992 214.151836 \n",
       "L 204.429886 213.897111 \n",
       "L 205.64978 212.938978 \n",
       "L 206.869675 213.928529 \n",
       "L 207.479622 213.744493 \n",
       "L 208.089569 214.450399 \n",
       "L 208.699516 213.912698 \n",
       "L 209.309463 213.025359 \n",
       "L 209.91941 213.061611 \n",
       "L 210.529358 213.51697 \n",
       "L 211.139305 214.299476 \n",
       "L 211.749252 212.86263 \n",
       "L 212.359199 213.053945 \n",
       "L 212.969146 213.689614 \n",
       "L 213.579093 213.639461 \n",
       "L 214.798988 213.96912 \n",
       "L 215.408935 213.83544 \n",
       "L 216.018882 213.861984 \n",
       "L 216.628829 213.618116 \n",
       "L 217.238776 213.809601 \n",
       "L 217.848724 213.808131 \n",
       "L 218.458671 212.937044 \n",
       "L 219.068618 213.636169 \n",
       "L 220.288512 214.165341 \n",
       "L 220.898459 213.917917 \n",
       "L 221.508407 214.219608 \n",
       "L 222.118354 213.052298 \n",
       "L 222.728301 213.382761 \n",
       "L 223.338248 212.924288 \n",
       "L 223.948195 213.945248 \n",
       "L 224.558142 213.094538 \n",
       "L 225.16809 213.548605 \n",
       "L 225.778037 213.693133 \n",
       "L 226.387984 213.464592 \n",
       "L 226.997931 214.108536 \n",
       "L 227.607878 214.07215 \n",
       "L 228.217825 213.36271 \n",
       "L 228.827773 212.153067 \n",
       "L 229.43772 213.802314 \n",
       "L 231.267561 212.752573 \n",
       "L 231.877508 213.37007 \n",
       "L 232.487456 212.344887 \n",
       "L 233.097403 212.581356 \n",
       "L 233.70735 213.818926 \n",
       "L 234.317297 213.392118 \n",
       "L 234.927244 213.359493 \n",
       "L 235.537191 212.659408 \n",
       "L 236.147139 213.265246 \n",
       "L 236.757086 213.209519 \n",
       "L 237.367033 212.000684 \n",
       "L 238.586927 213.543038 \n",
       "L 239.806822 213.94762 \n",
       "L 240.416769 212.843101 \n",
       "L 241.026716 213.657391 \n",
       "L 241.636663 213.53042 \n",
       "L 242.24661 213.671374 \n",
       "L 242.856557 213.51358 \n",
       "L 243.466505 213.127008 \n",
       "L 244.076452 213.399769 \n",
       "L 244.686399 213.098659 \n",
       "L 245.296346 212.220429 \n",
       "L 245.906293 213.24095 \n",
       "L 246.51624 212.992814 \n",
       "L 247.126188 213.050731 \n",
       "L 247.736135 213.85847 \n",
       "L 248.346082 213.201051 \n",
       "L 248.956029 213.562898 \n",
       "L 249.565976 212.651708 \n",
       "L 250.175923 212.931291 \n",
       "L 250.785871 213.375181 \n",
       "L 251.395818 213.431144 \n",
       "L 252.005765 213.237009 \n",
       "L 252.615712 212.475905 \n",
       "L 253.225659 213.278132 \n",
       "L 253.835606 213.161588 \n",
       "L 254.445554 213.289782 \n",
       "L 255.055501 213.82255 \n",
       "L 255.665448 213.115458 \n",
       "L 256.275395 213.293453 \n",
       "L 256.885342 212.464236 \n",
       "L 257.495289 213.462286 \n",
       "L 258.105237 212.158778 \n",
       "L 259.325131 213.048145 \n",
       "L 259.935078 212.860957 \n",
       "L 260.545025 212.848693 \n",
       "L 261.154972 212.661645 \n",
       "L 261.76492 213.404618 \n",
       "L 262.374867 212.103176 \n",
       "L 262.984814 212.70958 \n",
       "L 263.594761 213.049727 \n",
       "L 264.204708 212.958235 \n",
       "L 264.814655 212.733396 \n",
       "L 265.424603 213.001865 \n",
       "L 266.03455 212.728569 \n",
       "L 266.644497 213.024055 \n",
       "L 267.254444 211.79031 \n",
       "L 267.864391 213.475443 \n",
       "L 268.474338 212.232433 \n",
       "L 269.084286 212.463357 \n",
       "L 269.694233 212.872291 \n",
       "L 270.30418 212.427507 \n",
       "L 270.914127 212.685828 \n",
       "L 271.524074 213.22901 \n",
       "L 272.743969 212.712143 \n",
       "L 273.353916 212.823866 \n",
       "L 273.963863 213.178663 \n",
       "L 275.183757 213.015057 \n",
       "L 275.793704 213.274802 \n",
       "L 276.403652 212.352999 \n",
       "L 277.013599 212.610867 \n",
       "L 277.623546 212.659882 \n",
       "L 278.233493 212.914424 \n",
       "L 278.84344 212.860231 \n",
       "L 279.453387 211.764103 \n",
       "L 280.673282 213.346923 \n",
       "L 281.283229 212.450523 \n",
       "L 281.893176 213.515824 \n",
       "L 282.503123 212.309899 \n",
       "L 283.11307 212.597692 \n",
       "L 283.723018 213.077626 \n",
       "L 284.332965 212.51753 \n",
       "L 284.942912 212.796918 \n",
       "L 285.552859 212.801731 \n",
       "L 286.162806 211.722631 \n",
       "L 286.772753 213.434577 \n",
       "L 287.382701 212.601925 \n",
       "L 287.992648 213.372438 \n",
       "L 288.602595 213.342082 \n",
       "L 289.212542 213.081429 \n",
       "L 289.822489 213.056627 \n",
       "L 290.432436 211.503378 \n",
       "L 291.652331 212.788711 \n",
       "L 293.482172 212.504621 \n",
       "L 294.092119 213.029727 \n",
       "L 295.312014 212.321006 \n",
       "L 295.921961 211.8342 \n",
       "L 296.531908 212.923938 \n",
       "L 297.751802 212.336688 \n",
       "L 298.36175 212.46003 \n",
       "L 298.971697 212.841184 \n",
       "L 299.581644 212.212208 \n",
       "L 300.801538 212.747479 \n",
       "L 301.411485 212.155746 \n",
       "L 302.021433 212.821157 \n",
       "L 302.63138 212.071529 \n",
       "L 303.241327 213.105189 \n",
       "L 303.851274 212.211293 \n",
       "L 304.461221 213.118852 \n",
       "L 305.071168 212.280121 \n",
       "L 305.681116 213.125034 \n",
       "L 306.291063 211.915617 \n",
       "L 306.90101 212.865698 \n",
       "L 308.120904 213.001471 \n",
       "L 308.730851 211.89933 \n",
       "L 309.340799 212.899693 \n",
       "L 309.950746 212.735003 \n",
       "L 310.560693 212.176649 \n",
       "L 311.17064 211.424185 \n",
       "L 311.780587 212.18042 \n",
       "L 312.390534 212.352199 \n",
       "L 313.000482 212.017288 \n",
       "L 313.610429 212.214654 \n",
       "L 314.220376 212.600796 \n",
       "L 314.830323 212.675253 \n",
       "L 315.44027 213.081585 \n",
       "L 316.660165 212.09785 \n",
       "L 317.270112 212.744225 \n",
       "L 317.880059 212.665191 \n",
       "L 318.490006 212.05558 \n",
       "L 319.099953 211.889295 \n",
       "L 319.7099 212.130378 \n",
       "L 320.319848 211.921805 \n",
       "L 320.929795 212.709582 \n",
       "L 321.539742 212.737627 \n",
       "L 322.149689 212.572557 \n",
       "L 322.759636 212.195421 \n",
       "L 323.369583 212.738557 \n",
       "L 325.199425 212.347954 \n",
       "L 325.809372 212.400139 \n",
       "L 326.419319 210.996549 \n",
       "L 327.029266 212.429683 \n",
       "L 327.639214 211.180684 \n",
       "L 328.249161 212.636131 \n",
       "L 328.859108 211.944517 \n",
       "L 329.469055 212.068332 \n",
       "L 330.079002 211.833608 \n",
       "L 330.688949 212.890445 \n",
       "L 331.298897 212.704348 \n",
       "L 331.908844 211.627723 \n",
       "L 332.518791 211.972751 \n",
       "L 333.738685 212.239411 \n",
       "L 334.348632 212.708792 \n",
       "L 334.95858 212.7353 \n",
       "L 335.568527 212.358061 \n",
       "L 336.178474 212.469525 \n",
       "L 336.788421 211.944439 \n",
       "L 337.398368 211.616613 \n",
       "L 338.008315 210.859604 \n",
       "L 338.618263 212.018016 \n",
       "L 339.22821 212.283174 \n",
       "L 339.838157 211.925613 \n",
       "L 341.058051 211.562669 \n",
       "L 341.667998 212.803464 \n",
       "L 342.277946 212.214753 \n",
       "L 342.887893 211.969932 \n",
       "L 343.49784 211.354158 \n",
       "L 344.107787 212.08412 \n",
       "L 344.717734 212.140414 \n",
       "L 345.327681 212.766955 \n",
       "L 345.937629 212.477543 \n",
       "L 347.157523 211.126163 \n",
       "L 347.76747 212.247098 \n",
       "L 348.987365 212.151803 \n",
       "L 350.207259 211.638959 \n",
       "L 350.817206 211.928727 \n",
       "L 351.427153 212.050599 \n",
       "L 352.0371 212.585683 \n",
       "L 352.647048 212.040143 \n",
       "L 353.256995 212.043099 \n",
       "L 353.866942 212.356314 \n",
       "L 354.476889 211.720058 \n",
       "L 355.086836 211.946891 \n",
       "L 355.696783 211.856355 \n",
       "L 356.306731 212.033006 \n",
       "L 356.916678 211.679304 \n",
       "L 357.526625 212.082708 \n",
       "L 358.136572 211.621344 \n",
       "L 358.746519 212.151422 \n",
       "L 359.356466 211.790533 \n",
       "L 359.966414 212.169351 \n",
       "L 360.576361 211.449805 \n",
       "L 361.186308 211.5947 \n",
       "L 361.796255 212.279772 \n",
       "L 362.406202 212.140148 \n",
       "L 363.016149 210.98181 \n",
       "L 363.626097 212.129086 \n",
       "L 364.845991 211.428626 \n",
       "L 365.455938 212.090548 \n",
       "L 366.065885 211.519813 \n",
       "L 366.675832 211.86819 \n",
       "L 367.895727 211.448029 \n",
       "L 368.505674 211.988821 \n",
       "L 369.115621 212.180895 \n",
       "L 369.725568 212.24136 \n",
       "L 369.725568 212.24136 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 50.14375 225.607248 \n",
       "L 50.14375 8.167248 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 384.94375 225.607248 \n",
       "L 384.94375 8.167248 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 50.14375 225.607248 \n",
       "L 384.94375 225.607248 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 50.14375 8.167248 \n",
       "L 384.94375 8.167248 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pe4218c9b4f\">\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"50.14375\" y=\"8.167248\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 181 ms (started: 2021-07-24 15:39:39 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Validation MAE\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fallen-corporation",
   "metadata": {},
   "source": [
    "由于缩放问题，阅读该图可能有点困难：前几个周期的验证 MAE 显着高于随后的值。 让我们省略前 10 个数据点，它们的比例与曲线的其余部分不同："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "nervous-track",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg height=\"262.19625pt\" version=\"1.1\" viewBox=\"0 0 386.331023 262.19625\" width=\"386.331023pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2021-07-24T15:39:29.464709</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.4.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 262.19625 \n",
       "L 386.331023 262.19625 \n",
       "L 386.331023 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 43.78125 224.64 \n",
       "L 378.58125 224.64 \n",
       "L 378.58125 7.2 \n",
       "L 43.78125 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"m2dee987718\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.377011\" xlink:href=\"#m2dee987718\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(55.195761 239.238438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" id=\"DejaVuSans-30\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"120.619064\" xlink:href=\"#m2dee987718\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 100 -->\n",
       "      <g transform=\"translate(111.075314 239.238438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-31\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"182.861116\" xlink:href=\"#m2dee987718\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 200 -->\n",
       "      <g transform=\"translate(173.317366 239.238438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-32\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"245.103169\" xlink:href=\"#m2dee987718\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 300 -->\n",
       "      <g transform=\"translate(235.559419 239.238438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" id=\"DejaVuSans-33\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"307.345221\" xlink:href=\"#m2dee987718\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 400 -->\n",
       "      <g transform=\"translate(297.801471 239.238438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-34\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"369.587273\" xlink:href=\"#m2dee987718\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 500 -->\n",
       "      <g transform=\"translate(360.043523 239.238438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-35\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- Epochs -->\n",
       "     <g transform=\"translate(193.265625 252.916563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 628 4666 \n",
       "L 3578 4666 \n",
       "L 3578 4134 \n",
       "L 1259 4134 \n",
       "L 1259 2753 \n",
       "L 3481 2753 \n",
       "L 3481 2222 \n",
       "L 1259 2222 \n",
       "L 1259 531 \n",
       "L 3634 531 \n",
       "L 3634 0 \n",
       "L 628 0 \n",
       "L 628 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-45\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" id=\"DejaVuSans-70\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" id=\"DejaVuSans-6f\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" id=\"DejaVuSans-63\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" id=\"DejaVuSans-68\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" id=\"DejaVuSans-73\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-45\"/>\n",
       "      <use x=\"63.183594\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"126.660156\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"187.841797\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"242.822266\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"306.201172\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"me2804e49c9\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#me2804e49c9\" y=\"222.83579\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 2.2 -->\n",
       "      <g transform=\"translate(20.878125 226.635009)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" id=\"DejaVuSans-2e\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#me2804e49c9\" y=\"188.330523\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 2.4 -->\n",
       "      <g transform=\"translate(20.878125 192.129742)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#me2804e49c9\" y=\"153.825255\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 2.6 -->\n",
       "      <g transform=\"translate(20.878125 157.624474)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" id=\"DejaVuSans-36\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#me2804e49c9\" y=\"119.319988\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.8 -->\n",
       "      <g transform=\"translate(20.878125 123.119207)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" id=\"DejaVuSans-38\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#me2804e49c9\" y=\"84.81472\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 3.0 -->\n",
       "      <g transform=\"translate(20.878125 88.613939)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#me2804e49c9\" y=\"50.309453\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 3.2 -->\n",
       "      <g transform=\"translate(20.878125 54.108672)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#me2804e49c9\" y=\"15.804186\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 3.4 -->\n",
       "      <g transform=\"translate(20.878125 19.603404)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_15\">\n",
       "     <!-- Validation MAE -->\n",
       "     <g transform=\"translate(14.798438 153.093438)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 1831 0 \n",
       "L 50 4666 \n",
       "L 709 4666 \n",
       "L 2188 738 \n",
       "L 3669 4666 \n",
       "L 4325 4666 \n",
       "L 2547 0 \n",
       "L 1831 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-56\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" id=\"DejaVuSans-61\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-6c\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2906 2969 \n",
       "L 2906 4863 \n",
       "L 3481 4863 \n",
       "L 3481 0 \n",
       "L 2906 0 \n",
       "L 2906 525 \n",
       "Q 2725 213 2448 61 \n",
       "Q 2172 -91 1784 -91 \n",
       "Q 1150 -91 751 415 \n",
       "Q 353 922 353 1747 \n",
       "Q 353 2572 751 3078 \n",
       "Q 1150 3584 1784 3584 \n",
       "Q 2172 3584 2448 3432 \n",
       "Q 2725 3281 2906 2969 \n",
       "z\n",
       "M 947 1747 \n",
       "Q 947 1113 1208 752 \n",
       "Q 1469 391 1925 391 \n",
       "Q 2381 391 2643 752 \n",
       "Q 2906 1113 2906 1747 \n",
       "Q 2906 2381 2643 2742 \n",
       "Q 2381 3103 1925 3103 \n",
       "Q 1469 3103 1208 2742 \n",
       "Q 947 2381 947 1747 \n",
       "z\n",
       "\" id=\"DejaVuSans-64\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" id=\"DejaVuSans-74\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" id=\"DejaVuSans-6e\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 628 4666 \n",
       "L 1569 4666 \n",
       "L 2759 1491 \n",
       "L 3956 4666 \n",
       "L 4897 4666 \n",
       "L 4897 0 \n",
       "L 4281 0 \n",
       "L 4281 4097 \n",
       "L 3078 897 \n",
       "L 2444 897 \n",
       "L 1241 4097 \n",
       "L 1241 0 \n",
       "L 628 0 \n",
       "L 628 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-4d\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2188 4044 \n",
       "L 1331 1722 \n",
       "L 3047 1722 \n",
       "L 2188 4044 \n",
       "z\n",
       "M 1831 4666 \n",
       "L 2547 4666 \n",
       "L 4325 0 \n",
       "L 3669 0 \n",
       "L 3244 1197 \n",
       "L 1141 1197 \n",
       "L 716 0 \n",
       "L 50 0 \n",
       "L 1831 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-41\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-56\"/>\n",
       "      <use x=\"60.658203\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"121.9375\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"149.720703\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"177.503906\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"240.980469\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"302.259766\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"341.46875\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"369.251953\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"430.433594\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"493.8125\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"525.599609\" xlink:href=\"#DejaVuSans-4d\"/>\n",
       "      <use x=\"611.878906\" xlink:href=\"#DejaVuSans-41\"/>\n",
       "      <use x=\"680.287109\" xlink:href=\"#DejaVuSans-45\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_14\">\n",
       "    <path clip-path=\"url(#p2135834311)\" d=\"M 58.999432 17.083636 \n",
       "L 59.621852 41.44303 \n",
       "L 61.489114 85.08723 \n",
       "L 62.111534 88.074642 \n",
       "L 62.733955 105.678518 \n",
       "L 63.356375 111.401564 \n",
       "L 64.601217 133.348345 \n",
       "L 65.223637 138.644332 \n",
       "L 65.846058 146.560656 \n",
       "L 66.468478 146.911648 \n",
       "L 67.090899 156.478793 \n",
       "L 67.713319 152.873289 \n",
       "L 68.33574 159.337457 \n",
       "L 68.95816 149.712366 \n",
       "L 69.580581 171.359016 \n",
       "L 70.203001 163.638076 \n",
       "L 70.825422 157.971362 \n",
       "L 71.447842 173.327932 \n",
       "L 72.070263 167.211588 \n",
       "L 73.315104 176.956214 \n",
       "L 73.937524 163.105665 \n",
       "L 74.559945 173.332374 \n",
       "L 75.182365 172.029749 \n",
       "L 75.804786 172.663009 \n",
       "L 76.427206 168.314387 \n",
       "L 77.049627 166.185205 \n",
       "L 77.672048 181.336775 \n",
       "L 78.294468 185.540936 \n",
       "L 78.916889 176.587596 \n",
       "L 79.539309 180.093218 \n",
       "L 80.16173 184.671385 \n",
       "L 80.78415 185.953053 \n",
       "L 81.406571 188.499771 \n",
       "L 82.028991 186.427599 \n",
       "L 82.651412 185.304419 \n",
       "L 83.273832 185.954482 \n",
       "L 83.896253 180.682157 \n",
       "L 84.518673 177.150436 \n",
       "L 85.141094 184.93432 \n",
       "L 85.763514 195.006913 \n",
       "L 86.385935 186.056071 \n",
       "L 87.008355 196.22976 \n",
       "L 87.630776 195.717623 \n",
       "L 88.253196 181.417243 \n",
       "L 88.875617 191.60867 \n",
       "L 89.498038 177.069176 \n",
       "L 90.120458 192.131765 \n",
       "L 90.742879 195.27787 \n",
       "L 91.365299 191.96141 \n",
       "L 91.98772 194.608823 \n",
       "L 92.61014 193.558253 \n",
       "L 93.232561 198.00266 \n",
       "L 93.854981 197.900936 \n",
       "L 94.477402 183.780783 \n",
       "L 95.099822 197.820495 \n",
       "L 95.722243 201.460474 \n",
       "L 96.967084 195.409085 \n",
       "L 97.589504 197.313802 \n",
       "L 98.211925 201.973907 \n",
       "L 98.834345 198.409177 \n",
       "L 99.456766 197.277039 \n",
       "L 100.079186 205.465343 \n",
       "L 100.701607 199.891278 \n",
       "L 101.324027 203.725093 \n",
       "L 101.946448 203.626059 \n",
       "L 102.568869 203.367006 \n",
       "L 103.191289 205.393529 \n",
       "L 103.81371 188.344862 \n",
       "L 104.43613 199.370836 \n",
       "L 105.058551 197.947679 \n",
       "L 105.680971 204.616703 \n",
       "L 106.303392 191.893684 \n",
       "L 106.925812 201.138532 \n",
       "L 107.548233 198.412565 \n",
       "L 108.170653 199.653465 \n",
       "L 108.793074 195.492288 \n",
       "L 109.415494 197.823323 \n",
       "L 110.037915 203.223779 \n",
       "L 110.660335 199.785513 \n",
       "L 111.282756 204.250589 \n",
       "L 111.905176 196.87789 \n",
       "L 112.527597 202.014809 \n",
       "L 113.150017 197.475385 \n",
       "L 113.772438 195.798753 \n",
       "L 114.394858 194.852231 \n",
       "L 115.017279 198.85565 \n",
       "L 115.6397 206.129541 \n",
       "L 116.26212 206.652358 \n",
       "L 117.506961 191.71537 \n",
       "L 118.129382 209.155474 \n",
       "L 118.751802 205.010808 \n",
       "L 119.374223 203.200338 \n",
       "L 119.996643 204.66014 \n",
       "L 120.619064 201.674569 \n",
       "L 121.241484 204.021686 \n",
       "L 121.863905 192.750556 \n",
       "L 122.486325 199.934447 \n",
       "L 123.108746 181.927709 \n",
       "L 123.731166 189.863294 \n",
       "L 124.353587 196.116062 \n",
       "L 124.976007 193.246492 \n",
       "L 125.598428 210.393566 \n",
       "L 126.843269 207.843573 \n",
       "L 127.465689 211.584499 \n",
       "L 128.08811 205.723738 \n",
       "L 128.710531 196.67267 \n",
       "L 129.332951 213.854697 \n",
       "L 129.955372 208.412269 \n",
       "L 130.577792 213.109801 \n",
       "L 131.200213 205.992859 \n",
       "L 131.822633 212.003413 \n",
       "L 132.445054 208.096827 \n",
       "L 133.067474 194.817997 \n",
       "L 133.689895 192.025867 \n",
       "L 134.312315 206.155949 \n",
       "L 134.934736 205.485308 \n",
       "L 135.557156 201.661437 \n",
       "L 136.179577 207.085432 \n",
       "L 136.801997 197.747771 \n",
       "L 137.424418 201.162997 \n",
       "L 138.046838 198.291555 \n",
       "L 138.669259 211.216215 \n",
       "L 139.291679 206.451395 \n",
       "L 139.9141 197.397067 \n",
       "L 140.53652 196.650936 \n",
       "L 141.158941 199.523452 \n",
       "L 141.781362 214.756364 \n",
       "L 143.026203 205.752748 \n",
       "L 143.648623 197.086776 \n",
       "L 144.271044 211.759234 \n",
       "L 144.893464 198.291422 \n",
       "L 145.515885 206.077147 \n",
       "L 146.138305 208.026072 \n",
       "L 146.760726 207.696434 \n",
       "L 148.005567 199.743506 \n",
       "L 148.627987 193.189409 \n",
       "L 149.250408 203.504428 \n",
       "L 149.872828 204.905367 \n",
       "L 150.495249 199.578344 \n",
       "L 151.117669 209.756985 \n",
       "L 151.74009 208.175721 \n",
       "L 152.36251 204.721716 \n",
       "L 152.984931 192.173906 \n",
       "L 153.607352 211.055774 \n",
       "L 154.229772 208.014483 \n",
       "L 154.852193 201.930614 \n",
       "L 155.474613 206.502313 \n",
       "L 156.097034 199.317615 \n",
       "L 156.719454 197.738783 \n",
       "L 157.341875 191.799879 \n",
       "L 157.964295 195.32798 \n",
       "L 158.586716 196.527957 \n",
       "L 159.209136 202.61995 \n",
       "L 159.831557 201.619573 \n",
       "L 160.453977 198.055274 \n",
       "L 161.076398 205.013024 \n",
       "L 161.698818 205.007543 \n",
       "L 162.321239 194.194064 \n",
       "L 162.943659 202.125042 \n",
       "L 163.56608 205.721625 \n",
       "L 164.1885 197.390845 \n",
       "L 164.810921 186.725015 \n",
       "L 165.433341 187.521976 \n",
       "L 166.055762 202.080546 \n",
       "L 166.678183 191.632096 \n",
       "L 167.300603 201.657231 \n",
       "L 167.923024 193.214182 \n",
       "L 168.545444 204.125631 \n",
       "L 169.167865 176.395564 \n",
       "L 169.790285 205.321679 \n",
       "L 170.412706 202.125566 \n",
       "L 171.035126 193.504749 \n",
       "L 171.657547 197.04444 \n",
       "L 172.279967 195.342819 \n",
       "L 172.902388 198.57042 \n",
       "L 173.524808 187.581918 \n",
       "L 174.147229 197.78692 \n",
       "L 174.769649 191.005396 \n",
       "L 175.39207 189.908109 \n",
       "L 176.01449 201.343033 \n",
       "L 176.636911 178.232935 \n",
       "L 177.259331 193.239397 \n",
       "L 177.881752 188.673878 \n",
       "L 178.504172 194.559206 \n",
       "L 179.126593 190.734481 \n",
       "L 179.749014 192.822447 \n",
       "L 180.371434 191.616681 \n",
       "L 180.993855 199.042026 \n",
       "L 181.616275 191.008625 \n",
       "L 182.238696 204.494228 \n",
       "L 182.861116 191.13871 \n",
       "L 183.483537 193.953433 \n",
       "L 184.105957 190.437189 \n",
       "L 184.728378 192.203419 \n",
       "L 185.350798 196.408465 \n",
       "L 185.973219 192.644401 \n",
       "L 186.595639 185.609804 \n",
       "L 187.21806 180.379548 \n",
       "L 188.462901 189.278942 \n",
       "L 189.707742 193.361224 \n",
       "L 190.330162 183.974275 \n",
       "L 190.952583 184.321421 \n",
       "L 191.575003 182.729401 \n",
       "L 192.197424 195.187366 \n",
       "L 192.819845 181.808895 \n",
       "L 193.442265 191.240978 \n",
       "L 194.064686 189.066035 \n",
       "L 194.687106 187.430022 \n",
       "L 195.931947 173.095363 \n",
       "L 197.176788 187.900075 \n",
       "L 197.799209 185.146703 \n",
       "L 198.421629 195.707786 \n",
       "L 199.04405 187.663218 \n",
       "L 199.66647 174.387715 \n",
       "L 200.288891 174.930081 \n",
       "L 200.911311 181.742722 \n",
       "L 201.533732 193.449815 \n",
       "L 202.156152 171.953117 \n",
       "L 202.778573 174.81538 \n",
       "L 203.400993 184.325658 \n",
       "L 204.023414 183.575311 \n",
       "L 204.645834 186.797245 \n",
       "L 205.268255 188.50736 \n",
       "L 205.890676 186.507357 \n",
       "L 206.513096 186.904491 \n",
       "L 207.135517 183.255971 \n",
       "L 207.757937 186.120785 \n",
       "L 208.380358 186.098789 \n",
       "L 209.002778 173.066426 \n",
       "L 209.625199 183.526064 \n",
       "L 210.247619 188.029111 \n",
       "L 210.87004 191.443026 \n",
       "L 211.49246 187.7413 \n",
       "L 212.114881 192.254908 \n",
       "L 212.737301 174.790741 \n",
       "L 213.359722 179.734811 \n",
       "L 213.982142 172.875577 \n",
       "L 214.604563 188.150208 \n",
       "L 215.226983 175.422696 \n",
       "L 215.849404 182.216014 \n",
       "L 216.471824 184.378309 \n",
       "L 217.094245 180.959098 \n",
       "L 217.716666 190.593167 \n",
       "L 218.339086 190.048786 \n",
       "L 218.961507 179.434835 \n",
       "L 219.583927 161.337315 \n",
       "L 220.206348 186.01176 \n",
       "L 220.828768 181.372839 \n",
       "L 222.073609 170.306544 \n",
       "L 222.69603 179.544939 \n",
       "L 223.31845 164.207147 \n",
       "L 223.940871 167.744966 \n",
       "L 224.563291 186.260299 \n",
       "L 225.185712 179.874799 \n",
       "L 225.808132 179.386699 \n",
       "L 226.430553 168.912705 \n",
       "L 227.052973 177.976674 \n",
       "L 227.675394 177.142939 \n",
       "L 228.297814 159.057513 \n",
       "L 229.542655 182.132729 \n",
       "L 230.165076 184.842572 \n",
       "L 230.787497 188.185696 \n",
       "L 231.409917 171.660946 \n",
       "L 232.032338 183.843563 \n",
       "L 232.654758 181.943947 \n",
       "L 233.277179 184.052758 \n",
       "L 233.899599 181.692004 \n",
       "L 234.52202 175.908482 \n",
       "L 235.14444 179.989273 \n",
       "L 235.766861 175.484355 \n",
       "L 236.389281 162.345117 \n",
       "L 237.011702 177.613177 \n",
       "L 237.634122 173.900808 \n",
       "L 238.256543 174.767295 \n",
       "L 238.878963 186.851912 \n",
       "L 239.501384 177.016238 \n",
       "L 240.123804 182.429857 \n",
       "L 240.746225 168.7975 \n",
       "L 241.368645 172.980354 \n",
       "L 241.991066 179.621406 \n",
       "L 242.613486 180.458668 \n",
       "L 243.235907 177.554212 \n",
       "L 243.858328 166.167312 \n",
       "L 244.480748 178.169456 \n",
       "L 245.103169 176.425838 \n",
       "L 245.725589 178.343749 \n",
       "L 246.34801 186.314513 \n",
       "L 246.97043 175.73568 \n",
       "L 247.592851 178.398672 \n",
       "L 248.215271 165.992731 \n",
       "L 248.837692 180.924597 \n",
       "L 249.460112 161.42276 \n",
       "L 250.704953 174.728609 \n",
       "L 251.327374 171.928087 \n",
       "L 251.949794 171.744601 \n",
       "L 252.572215 168.946177 \n",
       "L 253.194635 180.061823 \n",
       "L 253.817056 160.590897 \n",
       "L 254.439476 169.663329 \n",
       "L 255.061897 174.752282 \n",
       "L 255.684317 173.383472 \n",
       "L 256.306738 170.019638 \n",
       "L 256.929159 174.036209 \n",
       "L 257.551579 169.947418 \n",
       "L 258.174 174.368208 \n",
       "L 258.79642 155.910092 \n",
       "L 259.418841 181.121431 \n",
       "L 260.041261 162.524716 \n",
       "L 260.663682 165.979579 \n",
       "L 261.286102 172.09766 \n",
       "L 261.908523 165.443219 \n",
       "L 262.530943 169.307967 \n",
       "L 263.153364 177.434545 \n",
       "L 263.775784 174.058555 \n",
       "L 264.398205 169.701676 \n",
       "L 265.020625 171.373166 \n",
       "L 265.643046 176.681298 \n",
       "L 266.887887 174.233588 \n",
       "L 267.510307 178.119633 \n",
       "L 268.132728 164.328501 \n",
       "L 268.755148 168.186483 \n",
       "L 269.377569 168.9198 \n",
       "L 269.99999 172.728 \n",
       "L 270.62241 171.917228 \n",
       "L 271.244831 155.518007 \n",
       "L 271.867251 168.69732 \n",
       "L 272.489672 179.198636 \n",
       "L 273.112092 165.787568 \n",
       "L 273.734513 181.725569 \n",
       "L 274.356933 163.683683 \n",
       "L 274.979354 167.989371 \n",
       "L 275.601774 175.169684 \n",
       "L 276.224195 166.790063 \n",
       "L 276.846615 170.969996 \n",
       "L 277.469036 171.042 \n",
       "L 278.091456 154.89755 \n",
       "L 278.713877 180.510034 \n",
       "L 279.336297 168.052696 \n",
       "L 279.958718 179.580376 \n",
       "L 280.581138 179.126211 \n",
       "L 281.203559 175.226581 \n",
       "L 281.82598 174.855506 \n",
       "L 282.4484 151.617298 \n",
       "L 283.070821 162.230035 \n",
       "L 283.693241 170.847212 \n",
       "L 284.315662 169.563601 \n",
       "L 284.938082 167.534599 \n",
       "L 285.560503 166.596931 \n",
       "L 286.182923 174.453056 \n",
       "L 287.427764 163.849862 \n",
       "L 288.050185 156.566736 \n",
       "L 288.672605 172.870353 \n",
       "L 289.917446 164.084477 \n",
       "L 290.539867 165.929797 \n",
       "L 291.162287 171.632255 \n",
       "L 291.784708 162.222128 \n",
       "L 292.407128 165.425912 \n",
       "L 293.029549 170.230334 \n",
       "L 293.651969 161.37739 \n",
       "L 294.27439 171.332639 \n",
       "L 294.896811 160.11742 \n",
       "L 295.519231 175.582047 \n",
       "L 296.141652 162.20844 \n",
       "L 296.764072 175.78646 \n",
       "L 297.386493 163.238176 \n",
       "L 298.008913 175.878948 \n",
       "L 298.631334 157.784822 \n",
       "L 299.253754 171.999012 \n",
       "L 299.876175 172.788384 \n",
       "L 300.498595 174.030327 \n",
       "L 301.121016 157.541147 \n",
       "L 301.743436 172.507617 \n",
       "L 302.365857 170.04369 \n",
       "L 302.988277 161.690128 \n",
       "L 303.610698 150.432479 \n",
       "L 304.233118 161.746542 \n",
       "L 304.855539 164.316531 \n",
       "L 305.477959 159.305928 \n",
       "L 306.10038 162.258716 \n",
       "L 306.7228 168.035801 \n",
       "L 307.345221 169.149757 \n",
       "L 307.967642 175.228916 \n",
       "L 308.590062 167.008358 \n",
       "L 309.212483 160.511211 \n",
       "L 309.834903 170.181652 \n",
       "L 310.457324 168.999229 \n",
       "L 311.079744 159.878804 \n",
       "L 311.702165 157.39102 \n",
       "L 312.324585 160.997872 \n",
       "L 312.947006 157.877403 \n",
       "L 313.569426 169.66336 \n",
       "L 314.191847 170.082942 \n",
       "L 314.814267 167.613319 \n",
       "L 315.436688 161.970977 \n",
       "L 316.059108 170.096855 \n",
       "L 317.303949 165.739894 \n",
       "L 317.92637 164.253032 \n",
       "L 318.54879 165.033776 \n",
       "L 319.171211 144.034608 \n",
       "L 319.793631 165.475786 \n",
       "L 320.416052 146.789461 \n",
       "L 321.038473 168.564458 \n",
       "L 321.660893 158.217186 \n",
       "L 322.283314 160.069592 \n",
       "L 322.905734 156.557872 \n",
       "L 323.528155 172.369254 \n",
       "L 324.150575 169.585052 \n",
       "L 324.772996 153.477622 \n",
       "L 325.395416 158.639596 \n",
       "L 326.017837 160.349371 \n",
       "L 326.640257 162.629113 \n",
       "L 327.262678 169.651544 \n",
       "L 327.885098 170.048133 \n",
       "L 328.507519 164.404238 \n",
       "L 329.129939 166.071851 \n",
       "L 329.75236 158.216024 \n",
       "L 330.37478 153.311412 \n",
       "L 330.997201 141.98577 \n",
       "L 331.619621 159.316818 \n",
       "L 332.242042 163.283855 \n",
       "L 332.864462 157.934373 \n",
       "L 333.486883 154.839181 \n",
       "L 334.109304 152.504352 \n",
       "L 334.731724 171.067935 \n",
       "L 335.354145 162.260197 \n",
       "L 335.976565 158.597424 \n",
       "L 336.598986 149.384809 \n",
       "L 337.221406 160.305801 \n",
       "L 337.843827 161.148009 \n",
       "L 338.466247 170.521713 \n",
       "L 339.088668 166.191807 \n",
       "L 339.711088 155.046031 \n",
       "L 340.333509 145.973774 \n",
       "L 340.955929 162.744122 \n",
       "L 341.57835 161.869614 \n",
       "L 342.20077 161.318404 \n",
       "L 343.445611 153.645724 \n",
       "L 344.068032 157.980957 \n",
       "L 344.690452 159.804291 \n",
       "L 345.312873 167.8097 \n",
       "L 345.935294 159.64785 \n",
       "L 346.557714 159.692079 \n",
       "L 347.180135 164.378108 \n",
       "L 347.802555 154.859059 \n",
       "L 348.424976 158.252705 \n",
       "L 349.047396 156.8982 \n",
       "L 349.669817 159.541078 \n",
       "L 350.292237 154.249327 \n",
       "L 350.914658 160.284679 \n",
       "L 351.537078 153.382192 \n",
       "L 352.159499 161.312707 \n",
       "L 352.781919 155.913434 \n",
       "L 353.40434 161.580939 \n",
       "L 354.02676 150.815792 \n",
       "L 354.649181 152.983568 \n",
       "L 355.271601 163.232952 \n",
       "L 355.894022 161.14404 \n",
       "L 356.516442 143.814091 \n",
       "L 357.138863 160.978529 \n",
       "L 357.761283 156.564505 \n",
       "L 358.383704 150.49893 \n",
       "L 359.006125 160.401961 \n",
       "L 359.628545 151.863184 \n",
       "L 360.250966 157.075259 \n",
       "L 361.495807 150.789209 \n",
       "L 362.118227 158.880022 \n",
       "L 362.740648 161.753648 \n",
       "L 363.363068 162.658266 \n",
       "L 363.363068 162.658266 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 43.78125 224.64 \n",
       "L 43.78125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 378.58125 224.64 \n",
       "L 378.58125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 43.78125 224.64 \n",
       "L 378.58125 224.64 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 43.78125 7.2 \n",
       "L 378.58125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p2135834311\">\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"43.78125\" y=\"7.2\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 423 ms (started: 2021-07-24 15:39:29 +08:00)\n"
     ]
    }
   ],
   "source": [
    "truncated_mae_history = average_mae_history[10:]\n",
    "plt.plot(range(1, len(truncated_mae_history) + 1), truncated_mae_history)\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Validation MAE\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "instructional-upset",
   "metadata": {},
   "source": [
    "根据此图，验证 MAE 在 120-140 个 epoch 后停止显着提高（这个数字包括我们省略的 10 个 epoch）。 过了那个点，你开始过度拟合。\n",
    "\n",
    "完成模型的其他参数调整后（除了 epoch 数，您还可以调整中间层的大小），您可以在所有训练数据上使用最佳参数训练最终生产模型 ，然后看它在测试数据上的表现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "ultimate-forwarding",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4/4 [==============================] - 0s 8ms/step - loss: 17.1650 - mae: 2.8036\n",
      "time: 27.7 s (started: 2021-07-24 15:38:04 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = build_model()\n",
    "\n",
    "model.fit(train_data, train_targets, epochs=130, batch_size=16, verbose=0)\n",
    "\n",
    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "electronic-rochester",
   "metadata": {},
   "source": [
    "这是最终结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "quarterly-color",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.803586721420288"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.51 ms (started: 2021-07-24 15:42:34 +08:00)\n"
     ]
    }
   ],
   "source": [
    "test_mae_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "supreme-worse",
   "metadata": {},
   "source": [
    "你仍然低了大约 2500 美元。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unavailable-device",
   "metadata": {},
   "source": [
    "### 生成对新数据的预测"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "blond-architecture",
   "metadata": {},
   "source": [
    "在我们的二元分类模型上调用 predict() 时，我们为每个输入样本检索了 0 到 1 之间的标量分数。 使用我们的多类分类模型，我们为每个样本检索了所有类的概率分布。 现在，使用这个标量回归模型， predict() 返回模型对样本价格的数千美元的猜测："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "suburban-india",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 143 ms (started: 2021-07-24 15:45:26 +08:00)\n"
     ]
    }
   ],
   "source": [
    "predictions = model.predict(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "informational-lexington",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([8.6478815], dtype=float32)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.42 ms (started: 2021-07-24 15:45:33 +08:00)\n"
     ]
    }
   ],
   "source": [
    "predictions[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "integral-poison",
   "metadata": {},
   "source": [
    "测试集中的第一栋房子的价格预计约为 10,000 美元。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "surprised-genome",
   "metadata": {},
   "source": [
    "### 尝试使用autokeras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "marine-accreditation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.66 s (started: 2021-07-25 12:40:00 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import autokeras as ak"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "agricultural-termination",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 5.99 ms (started: 2021-07-25 12:37:50 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import boston_housing\n",
    "\n",
    "(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "early-sandwich",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(404, 13)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.76 ms (started: 2021-07-25 12:39:07 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "arbitrary-sydney",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(404,)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.41 ms (started: 2021-07-25 12:41:54 +08:00)\n"
     ]
    }
   ],
   "source": [
    "train_targets.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "backed-memorial",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['BinaryCrossentropy',\n",
       " 'CategoricalCrossentropy',\n",
       " 'CategoricalHinge',\n",
       " 'CosineSimilarity',\n",
       " 'Hinge',\n",
       " 'Huber',\n",
       " 'KLD',\n",
       " 'KLDivergence',\n",
       " 'LogCosh',\n",
       " 'Loss',\n",
       " 'MAE',\n",
       " 'MAPE',\n",
       " 'MSE',\n",
       " 'MSLE',\n",
       " 'MeanAbsoluteError',\n",
       " 'MeanAbsolutePercentageError',\n",
       " 'MeanSquaredError',\n",
       " 'MeanSquaredLogarithmicError',\n",
       " 'Poisson',\n",
       " 'Reduction',\n",
       " 'SparseCategoricalCrossentropy',\n",
       " 'SquaredHinge',\n",
       " '__builtins__',\n",
       " '__cached__',\n",
       " '__doc__',\n",
       " '__file__',\n",
       " '__loader__',\n",
       " '__name__',\n",
       " '__package__',\n",
       " '__path__',\n",
       " '__spec__',\n",
       " '_sys',\n",
       " 'binary_crossentropy',\n",
       " 'categorical_crossentropy',\n",
       " 'categorical_hinge',\n",
       " 'cosine_similarity',\n",
       " 'deserialize',\n",
       " 'get',\n",
       " 'hinge',\n",
       " 'huber',\n",
       " 'kl_divergence',\n",
       " 'kld',\n",
       " 'kullback_leibler_divergence',\n",
       " 'log_cosh',\n",
       " 'logcosh',\n",
       " 'mae',\n",
       " 'mape',\n",
       " 'mean_absolute_error',\n",
       " 'mean_absolute_percentage_error',\n",
       " 'mean_squared_error',\n",
       " 'mean_squared_logarithmic_error',\n",
       " 'mse',\n",
       " 'msle',\n",
       " 'poisson',\n",
       " 'serialize',\n",
       " 'sparse_categorical_crossentropy',\n",
       " 'squared_hinge']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4.52 ms (started: 2021-07-25 13:20:12 +08:00)\n"
     ]
    }
   ],
   "source": [
    "dir(tf.keras.losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "municipal-lover",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 11 ms (started: 2021-07-25 13:22:14 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# Initialize the structured data regressor.\n",
    "reg = ak.StructuredDataRegressor(\n",
    "    overwrite=True, max_trials=5, loss=\"mse\", metrics=\"mae\"\n",
    ")  # It tries 3 different models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "corporate-somerset",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 5 Complete [00h 00m 53s]\n",
      "val_loss: 7.9547929763793945\n",
      "\n",
      "Best val_loss So Far: 5.462915897369385\n",
      "Total elapsed time: 00h 06m 25s\n",
      "INFO:tensorflow:Oracle triggered exit\n",
      "Epoch 1/250\n",
      "13/13 [==============================] - 1s 17ms/step - loss: 574.1790 - mae: 22.0731\n",
      "Epoch 2/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 540.6957 - mae: 21.3892 0s - loss: 512.0692 - mae: 20.\n",
      "Epoch 3/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 504.0912 - mae: 20.6075\n",
      "Epoch 4/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 460.4326 - mae: 19.6278\n",
      "Epoch 5/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 407.4809 - mae: 18.3676\n",
      "Epoch 6/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 344.5948 - mae: 16.7594\n",
      "Epoch 7/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 273.5533 - mae: 14.7213\n",
      "Epoch 8/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 201.5718 - mae: 12.2677\n",
      "Epoch 9/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 140.2464 - mae: 9.7691\n",
      "Epoch 10/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 98.5851 - mae: 7.9096\n",
      "Epoch 11/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 75.3013 - mae: 6.7353\n",
      "Epoch 12/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 61.6399 - mae: 5.9617\n",
      "Epoch 13/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 51.7789 - mae: 5.3724\n",
      "Epoch 14/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 44.3387 - mae: 4.8957\n",
      "Epoch 15/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 38.7797 - mae: 4.5327\n",
      "Epoch 16/250\n",
      "13/13 [==============================] - 0s 19ms/step - loss: 34.5528 - mae: 4.2456\n",
      "Epoch 17/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 31.2995 - mae: 4.0244\n",
      "Epoch 18/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 28.7739 - mae: 3.8510\n",
      "Epoch 19/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 26.7646 - mae: 3.7076\n",
      "Epoch 20/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 25.1207 - mae: 3.5832\n",
      "Epoch 21/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 23.7575 - mae: 3.4762\n",
      "Epoch 22/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 22.6025 - mae: 3.3830\n",
      "Epoch 23/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 21.6045 - mae: 3.3018\n",
      "Epoch 24/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 20.7322 - mae: 3.2275\n",
      "Epoch 25/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 19.9671 - mae: 3.1614\n",
      "Epoch 26/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 19.2826 - mae: 3.1007\n",
      "Epoch 27/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 18.6650 - mae: 3.0445\n",
      "Epoch 28/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 18.1125 - mae: 2.9966\n",
      "Epoch 29/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 17.6089 - mae: 2.9536\n",
      "Epoch 30/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 17.1505 - mae: 2.9137\n",
      "Epoch 31/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 16.7290 - mae: 2.8747\n",
      "Epoch 32/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 16.3396 - mae: 2.8382\n",
      "Epoch 33/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 15.9708 - mae: 2.8045\n",
      "Epoch 34/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 15.6226 - mae: 2.7734\n",
      "Epoch 35/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 15.2963 - mae: 2.7443\n",
      "Epoch 36/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 14.9904 - mae: 2.7175\n",
      "Epoch 37/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 14.7014 - mae: 2.6924\n",
      "Epoch 38/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 14.4272 - mae: 2.6676\n",
      "Epoch 39/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 14.1633 - mae: 2.6444\n",
      "Epoch 40/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 13.9071 - mae: 2.6219\n",
      "Epoch 41/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 13.6559 - mae: 2.6004\n",
      "Epoch 42/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 13.4127 - mae: 2.5798\n",
      "Epoch 43/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 13.1825 - mae: 2.5606\n",
      "Epoch 44/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 12.9712 - mae: 2.5425\n",
      "Epoch 45/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 12.7698 - mae: 2.5259\n",
      "Epoch 46/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 12.5819 - mae: 2.5095\n",
      "Epoch 47/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 12.3980 - mae: 2.4923\n",
      "Epoch 48/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 12.2215 - mae: 2.4757\n",
      "Epoch 49/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 12.0508 - mae: 2.4594\n",
      "Epoch 50/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 11.8903 - mae: 2.4449\n",
      "Epoch 51/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 11.7373 - mae: 2.4310\n",
      "Epoch 52/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 11.5917 - mae: 2.4171\n",
      "Epoch 53/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 11.4542 - mae: 2.4048\n",
      "Epoch 54/250\n",
      "13/13 [==============================] - 0s 18ms/step - loss: 11.3226 - mae: 2.3932\n",
      "Epoch 55/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 11.1997 - mae: 2.3817\n",
      "Epoch 56/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 11.0806 - mae: 2.3709\n",
      "Epoch 57/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 10.9685 - mae: 2.3607\n",
      "Epoch 58/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 10.8606 - mae: 2.3504\n",
      "Epoch 59/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 10.7530 - mae: 2.3407\n",
      "Epoch 60/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 10.6517 - mae: 2.3321\n",
      "Epoch 61/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 10.5566 - mae: 2.3235\n",
      "Epoch 62/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 10.4628 - mae: 2.3157\n",
      "Epoch 63/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 10.3775 - mae: 2.3078\n",
      "Epoch 64/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 10.2883 - mae: 2.2997\n",
      "Epoch 65/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 10.2029 - mae: 2.2923\n",
      "Epoch 66/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 10.1250 - mae: 2.2844\n",
      "Epoch 67/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 10.0473 - mae: 2.2773\n",
      "Epoch 68/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 9.9740 - mae: 2.2705\n",
      "Epoch 69/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 9.9033 - mae: 2.2636\n",
      "Epoch 70/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 9.8367 - mae: 2.2563\n",
      "Epoch 71/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 9.7696 - mae: 2.2494\n",
      "Epoch 72/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.7074 - mae: 2.2424\n",
      "Epoch 73/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.6469 - mae: 2.2355\n",
      "Epoch 74/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.5860 - mae: 2.2281\n",
      "Epoch 75/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.5288 - mae: 2.2211\n",
      "Epoch 76/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.4722 - mae: 2.2139\n",
      "Epoch 77/250\n",
      "13/13 [==============================] - 0s 21ms/step - loss: 9.4168 - mae: 2.2074\n",
      "Epoch 78/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 9.3638 - mae: 2.2012\n",
      "Epoch 79/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.3135 - mae: 2.1948\n",
      "Epoch 80/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 9.2634 - mae: 2.1875\n",
      "Epoch 81/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.2138 - mae: 2.1820\n",
      "Epoch 82/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.1658 - mae: 2.1751\n",
      "Epoch 83/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 9.1196 - mae: 2.1695\n",
      "Epoch 84/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.0734 - mae: 2.1626\n",
      "Epoch 85/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 9.0254 - mae: 2.1566\n",
      "Epoch 86/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.9841 - mae: 2.1508\n",
      "Epoch 87/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.9381 - mae: 2.1450\n",
      "Epoch 88/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.9010 - mae: 2.1402\n",
      "Epoch 89/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.8587 - mae: 2.1341\n",
      "Epoch 90/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.8146 - mae: 2.1289\n",
      "Epoch 91/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.7725 - mae: 2.1229\n",
      "Epoch 92/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.7262 - mae: 2.1171\n",
      "Epoch 93/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.6843 - mae: 2.1114\n",
      "Epoch 94/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.6394 - mae: 2.1049\n",
      "Epoch 95/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.5987 - mae: 2.0993\n",
      "Epoch 96/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.5586 - mae: 2.0934\n",
      "Epoch 97/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 8.5192 - mae: 2.0884\n",
      "Epoch 98/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.4836 - mae: 2.0834\n",
      "Epoch 99/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.4461 - mae: 2.0782\n",
      "Epoch 100/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.4134 - mae: 2.0738\n",
      "Epoch 101/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.3783 - mae: 2.0686\n",
      "Epoch 102/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.3421 - mae: 2.0639\n",
      "Epoch 103/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.3126 - mae: 2.0594\n",
      "Epoch 104/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.2794 - mae: 2.0551\n",
      "Epoch 105/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 8.2502 - mae: 2.0506\n",
      "Epoch 106/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.2172 - mae: 2.0464\n",
      "Epoch 107/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.1914 - mae: 2.0422\n",
      "Epoch 108/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.1595 - mae: 2.0376\n",
      "Epoch 109/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.1344 - mae: 2.0344\n",
      "Epoch 110/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 8.1029 - mae: 2.0302\n",
      "Epoch 111/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.0716 - mae: 2.0267\n",
      "Epoch 112/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 8.0391 - mae: 2.0231\n",
      "Epoch 113/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 8.0042 - mae: 2.0192\n",
      "Epoch 114/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.9732 - mae: 2.0158\n",
      "Epoch 115/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 7.9386 - mae: 2.0113\n",
      "Epoch 116/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.9048 - mae: 2.0071\n",
      "Epoch 117/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 7.8725 - mae: 2.0029\n",
      "Epoch 118/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 7.8405 - mae: 1.9991\n",
      "Epoch 119/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.8088 - mae: 1.9950\n",
      "Epoch 120/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.7805 - mae: 1.9919\n",
      "Epoch 121/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.7503 - mae: 1.9880\n",
      "Epoch 122/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 7.7221 - mae: 1.9846\n",
      "Epoch 123/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.6920 - mae: 1.9808\n",
      "Epoch 124/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 7.6680 - mae: 1.9779\n",
      "Epoch 125/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.6436 - mae: 1.9747\n",
      "Epoch 126/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 7.6172 - mae: 1.9708\n",
      "Epoch 127/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 7.5908 - mae: 1.9674\n",
      "Epoch 128/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.5663 - mae: 1.9637\n",
      "Epoch 129/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.5438 - mae: 1.9613\n",
      "Epoch 130/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.5173 - mae: 1.9572\n",
      "Epoch 131/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.4960 - mae: 1.9549\n",
      "Epoch 132/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.4720 - mae: 1.9512\n",
      "Epoch 133/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.4483 - mae: 1.9475\n",
      "Epoch 134/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 7.4255 - mae: 1.9447\n",
      "Epoch 135/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.4047 - mae: 1.9416\n",
      "Epoch 136/250\n",
      "13/13 [==============================] - 0s 18ms/step - loss: 7.3806 - mae: 1.9380\n",
      "Epoch 137/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.3613 - mae: 1.9357\n",
      "Epoch 138/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 7.3358 - mae: 1.9318\n",
      "Epoch 139/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 7.3168 - mae: 1.9290\n",
      "Epoch 140/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 7.2939 - mae: 1.9259\n",
      "Epoch 141/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 7.2698 - mae: 1.9217\n",
      "Epoch 142/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 7.2487 - mae: 1.9194\n",
      "Epoch 143/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.2232 - mae: 1.9151\n",
      "Epoch 144/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.2018 - mae: 1.9127\n",
      "Epoch 145/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.1801 - mae: 1.9086\n",
      "Epoch 146/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.1524 - mae: 1.9052\n",
      "Epoch 147/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 7.1260 - mae: 1.9009\n",
      "Epoch 148/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.1034 - mae: 1.8977\n",
      "Epoch 149/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.0732 - mae: 1.8933\n",
      "Epoch 150/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.0542 - mae: 1.8907\n",
      "Epoch 151/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.0279 - mae: 1.8866\n",
      "Epoch 152/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 7.0052 - mae: 1.8836\n",
      "Epoch 153/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 6.9821 - mae: 1.8803\n",
      "Epoch 154/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.9558 - mae: 1.8766\n",
      "Epoch 155/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.9376 - mae: 1.8739\n",
      "Epoch 156/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.9110 - mae: 1.8708\n",
      "Epoch 157/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.8856 - mae: 1.8667\n",
      "Epoch 158/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.8669 - mae: 1.8648\n",
      "Epoch 159/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.8437 - mae: 1.8612\n",
      "Epoch 160/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.8198 - mae: 1.8582\n",
      "Epoch 161/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.7975 - mae: 1.8550\n",
      "Epoch 162/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.7803 - mae: 1.8526\n",
      "Epoch 163/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.7537 - mae: 1.8488\n",
      "Epoch 164/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.7302 - mae: 1.8459\n",
      "Epoch 165/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.7131 - mae: 1.8439\n",
      "Epoch 166/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 6.6895 - mae: 1.8403\n",
      "Epoch 167/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.6695 - mae: 1.8379\n",
      "Epoch 168/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.6484 - mae: 1.8349\n",
      "Epoch 169/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.6290 - mae: 1.8323\n",
      "Epoch 170/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.6062 - mae: 1.8294\n",
      "Epoch 171/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.5905 - mae: 1.8271\n",
      "Epoch 172/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 6.5648 - mae: 1.8225\n",
      "Epoch 173/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.5476 - mae: 1.8204\n",
      "Epoch 174/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.5253 - mae: 1.8175\n",
      "Epoch 175/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 6.5034 - mae: 1.8151\n",
      "Epoch 176/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.4838 - mae: 1.8124\n",
      "Epoch 177/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.4623 - mae: 1.8093\n",
      "Epoch 178/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.4430 - mae: 1.8069\n",
      "Epoch 179/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.4251 - mae: 1.8044\n",
      "Epoch 180/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.3987 - mae: 1.8004\n",
      "Epoch 181/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.3867 - mae: 1.7986\n",
      "Epoch 182/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.3644 - mae: 1.7955\n",
      "Epoch 183/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.3453 - mae: 1.7924\n",
      "Epoch 184/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.3268 - mae: 1.7906\n",
      "Epoch 185/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.3085 - mae: 1.7880\n",
      "Epoch 186/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.2862 - mae: 1.7845\n",
      "Epoch 187/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.2670 - mae: 1.7820\n",
      "Epoch 188/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 6.2453 - mae: 1.7785: 0s - loss: 5.2762 - mae: 1.73\n",
      "Epoch 189/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.2277 - mae: 1.7765\n",
      "Epoch 190/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 6.2052 - mae: 1.7729\n",
      "Epoch 191/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 6.1904 - mae: 1.7711\n",
      "Epoch 192/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.1657 - mae: 1.7668\n",
      "Epoch 193/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.1534 - mae: 1.7659\n",
      "Epoch 194/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.1336 - mae: 1.7627\n",
      "Epoch 195/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.1136 - mae: 1.7600\n",
      "Epoch 196/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.0941 - mae: 1.7571\n",
      "Epoch 197/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.0745 - mae: 1.7542\n",
      "Epoch 198/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 6.0612 - mae: 1.7522\n",
      "Epoch 199/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.0384 - mae: 1.7479\n",
      "Epoch 200/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 6.0200 - mae: 1.7444\n",
      "Epoch 201/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.9982 - mae: 1.7415\n",
      "Epoch 202/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.9864 - mae: 1.7406\n",
      "Epoch 203/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.9636 - mae: 1.7370\n",
      "Epoch 204/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.9441 - mae: 1.7343\n",
      "Epoch 205/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.9284 - mae: 1.7318\n",
      "Epoch 206/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.9119 - mae: 1.7305\n",
      "Epoch 207/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.8934 - mae: 1.7273\n",
      "Epoch 208/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.8730 - mae: 1.7243\n",
      "Epoch 209/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.8610 - mae: 1.7229\n",
      "Epoch 210/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.8403 - mae: 1.7192\n",
      "Epoch 211/250\n",
      "13/13 [==============================] - 0s 18ms/step - loss: 5.8221 - mae: 1.7175\n",
      "Epoch 212/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.8045 - mae: 1.7155\n",
      "Epoch 213/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.7891 - mae: 1.7136\n",
      "Epoch 214/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.7648 - mae: 1.7094\n",
      "Epoch 215/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.7479 - mae: 1.7081\n",
      "Epoch 216/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.7362 - mae: 1.7064\n",
      "Epoch 217/250\n",
      "13/13 [==============================] - 0s 20ms/step - loss: 5.7095 - mae: 1.7019\n",
      "Epoch 218/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.6967 - mae: 1.7020\n",
      "Epoch 219/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.6759 - mae: 1.6990\n",
      "Epoch 220/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.6690 - mae: 1.6964\n",
      "Epoch 221/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.6430 - mae: 1.6919\n",
      "Epoch 222/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.6269 - mae: 1.6914\n",
      "Epoch 223/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.6142 - mae: 1.6897\n",
      "Epoch 224/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.5914 - mae: 1.6854\n",
      "Epoch 225/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.5793 - mae: 1.6844\n",
      "Epoch 226/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.5599 - mae: 1.6813\n",
      "Epoch 227/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.5435 - mae: 1.6787\n",
      "Epoch 228/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.5285 - mae: 1.6774\n",
      "Epoch 229/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.5080 - mae: 1.6744\n",
      "Epoch 230/250\n",
      "13/13 [==============================] - 0s 19ms/step - loss: 5.4943 - mae: 1.6718\n",
      "Epoch 231/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.4738 - mae: 1.6692\n",
      "Epoch 232/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.4622 - mae: 1.6669\n",
      "Epoch 233/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.4412 - mae: 1.6645\n",
      "Epoch 234/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.4296 - mae: 1.6630\n",
      "Epoch 235/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.4022 - mae: 1.6572\n",
      "Epoch 236/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.3929 - mae: 1.6562\n",
      "Epoch 237/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.3697 - mae: 1.6530\n",
      "Epoch 238/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.3556 - mae: 1.6513\n",
      "Epoch 239/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.3340 - mae: 1.6482\n",
      "Epoch 240/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.3178 - mae: 1.6461\n",
      "Epoch 241/250\n",
      "13/13 [==============================] - 0s 18ms/step - loss: 5.2970 - mae: 1.6418\n",
      "Epoch 242/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.2793 - mae: 1.6401\n",
      "Epoch 243/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.2668 - mae: 1.6396\n",
      "Epoch 244/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.2523 - mae: 1.6360\n",
      "Epoch 245/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.2303 - mae: 1.6340\n",
      "Epoch 246/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.2166 - mae: 1.6328\n",
      "Epoch 247/250\n",
      "13/13 [==============================] - 0s 17ms/step - loss: 5.1971 - mae: 1.6292\n",
      "Epoch 248/250\n",
      "13/13 [==============================] - 0s 16ms/step - loss: 5.1831 - mae: 1.6278\n",
      "Epoch 249/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.1642 - mae: 1.6256\n",
      "Epoch 250/250\n",
      "13/13 [==============================] - 0s 15ms/step - loss: 5.1515 - mae: 1.6247\n",
      "INFO:tensorflow:Assets written to: ./structured_data_regressor/best_model/assets\n",
      "time: 7min 41s (started: 2021-07-25 13:22:17 +08:00)\n"
     ]
    }
   ],
   "source": [
    "reg.fit(x=train_data, y = train_targets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "thrown-mount",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         [(None, 13)]              0         \n",
      "_________________________________________________________________\n",
      "multi_category_encoding (Mul (None, 13)                0         \n",
      "_________________________________________________________________\n",
      "normalization (Normalization (None, 13)                27        \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 32)                448       \n",
      "_________________________________________________________________\n",
      "re_lu (ReLU)                 (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 32)                1056      \n",
      "_________________________________________________________________\n",
      "re_lu_1 (ReLU)               (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "regression_head_1 (Dense)    (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,564\n",
      "Trainable params: 1,537\n",
      "Non-trainable params: 27\n",
      "_________________________________________________________________\n",
      "time: 941 ms (started: 2021-07-25 13:52:32 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = reg.export_model()\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "stuffed-negotiation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 480 µs (started: 2021-07-25 13:16:47 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.utils import plot_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "collectible-bloom",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzEAAANQCAIAAACXXHYoAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdaVwUV74//tMs3TTd0CA7iIpMCK9LtCXgKArjgkNHQQwExCgmmQxefsaIRHEE45LrEqIxKjNuKDGOQRRiXjoDihlDNHNFTMAE1KhgUFQ2BZGGRmxE6v/g/KdupYGm2bpAPu9H9qlTVd86lPSXqrMIGIYhAAAAAMArA74DAAAAAADkZAAAAAADAHIyAAAAAP4hJwMAAADgnxHfAYCu8vLytm/fzncUAAAwmPj4+CxfvpzvKEAneE42aNy/f//48eN8RwFD1/Hjx8vLy/mOYkC4dOnSpUuX+I5iAMG9MWBdunQpLy+P7yhAV3hONsh89dVXfIcAQ5RAIPjggw/mzp3LdyD8Cw8PJ/jPyIF7Y8Ci9yoMFnhOBgAAAMA/5GQAAAAA/ENOBgAAAMA/5GQAAAAA/ENOBgAA/S41NVXwH1KpVGPr3bt3g4ODGxoaamtr2Wqenp5Pnz7lVuNuFQgE3t7eeryC7jl9+rSbm5uRUQcD6RiGyc3NXbJkiZubm0gksrW19fX1TU1N5S4/HR8fn56errFjfHw8e+0TJ07s3wsAPiAnAwDQE5VK9dJLLwUFBfEdCG/27t3LMIxKpeIWFhYWent7BwQEmJubW1tbMwyTn59Py2NjY7k16da8vDwrKyuGYQoKCvQavW5KS0uDg4MTEhIePHjQYYXi4mJfX9+SkpLjx48rlcpLly6NGDFi4cKFK1euZOssWrQoISFh7dq13B0/+eQThmEYhjE0NOzfawCeICcDANAThmHa2tra2tr4CkAqlfr6+vJ19g41NDTMnj37jTfeeP/997nlIpHIysoqOTn56NGjfMXWM2vXrp00adLly5fNzMw6q2NkZJSRkTF27FgTE5PRo0cfOnTIyspq165darWaVnB1dT1x4sTmzZszMjL0FTjwDzkZAICemJmZlZaWnj59mu9ABpCtW7dWV1evW7dOo9zExOTIkSMGBgbR0dElJSW8xNYzn3/+eXx8fIdvLSl3d/dnz55ZWlqyJUKh0NnZWa1Wc9/VyuXysLCwFStWtLa29m/EMGAgJwMAAH4wDJOSkjJhwgRHR8f2WxUKxZo1axobG8PDwzU6lg1kYrG4u7vU19ffunXL09NTJpNxy0NCQsrLy0+dOtV30cGAhpwMAEAfTp48yXbQphkGt6SsrCwiIsLCwsLKyiooKKi0tJTutW3bNlph+PDh+fn5/v7+ZmZmpqam06ZNy83NpXU2bdpE67DvJc+cOUNLrK2tucdpamrKzc2lm7Q8yNGboqKiBw8eyOXyziqsX78+ICDgypUrS5cu1X6oR48eLV++3NXVVSgUWlpazpw589y5c3STLu1M1dTUxMTEjBo1SigU2tjYhIaGFhYW9v4ytWhoaMjNzQ0ODra3tz98+LDG1nHjxhFCvvnmm36NAQYO5GQAAPrw+uuvMwwzZ86cDktiY2NjY2MrKirS09O/++67N998k9aJi4tjGEYul9fX1y9btmzTpk3V1dX//ve/6+rqpk+f/v333xNC1qxZwzCMRCJhj/zaa68xDOPl5cWW0ONIJJLJkyfTfuLcN2LTp0+3srLS/yKe165dI4QMHz68swoGBgZHjhxxdnZOSUk5cuRIZ9Wqq6vHjx+flpaWlJRUW1v7ww8/mJqa+vv7p6SkEN3amRBSVVU1fvz4jIyMPXv21NXVnT9/vq6uzsfHp//Wi9y0aZNMJvP19TU0NDxx4sQrr7yiUcHJyYn8p5VgKEBOBgDAv6ioKB8fH4lEMmPGjMDAwPz8/NraWm6FpqamPXv20Dre3t6pqaktLS3Lli3rk7O3tbXRRK1Pjqa7qqoqQojGCzsN1tbWGRkZxsbG0dHRN2/e7LBOQkLCnTt3du7cGRQUZG5u7ubmlpaW5uDgEBMTozH4UUs7JyQk3L17d/v27bNmzZJKpR4eHseOHWMYpstHdD22Zs0atVp948YNd3d3T0/PjRs3alQwNzcXCAS0lWAoQE4GAMC/8ePHs/92dnYmhFRWVnIrSCQS+iaLGjNmjKOjY1FRUZ98YbPPhHp/qG6h73CNjY21V5s4ceK2bduamprCw8Obm5vbVzhx4gQhJDAwkC0RiUT+/v7Nzc0aL/60tPPJkycNDAy4M5XY29t7eHhcvny5vLy8u5emI6FQ6O7uvnfv3uDg4HXr1n377bcaFYyMjDq8ZHghIScDAOAf91mRUCgkhGhMmWFhYaGxi62tLSHk4cOH/R9dfzExMSGEPHv2rMuaMTExERER165d05gygxCiVquVSqWJiYnG3BN2dnaEkOrqam5hZ+1MD9LW1iaTybjT0v7000+EkFu3bvXsAnU3e/ZsQkhWVpZGeWtraw8GDcAgxX8fTwAA6NKjR48YhhEIBGwJzcZoZkYIMTAwaGlp4e5SX1+vcRDu7gOBg4MDIUSpVOpSOSUlpbCw8ODBgzSTY4lEIplMplQqGxsbuWkZfWtpb2+vy8FFIpGFhYVKpWpubuZl9INIJCKE1NXVcQsbGhoYhqGtBEMBnpMBAAwCT58+pbPbU1evXq2srJTL5ewXtoODQ0VFBVuhurr63r17GgcxNTVl87aXX355//79/Rx1F2ivdh3fDEql0q+//loikezZs0djU0hICCGEO2eEWq3OyckRi8UKhULHYEJDQ1tbW9nRrNSWLVtGjBjRtzOExcXFRUZGahRmZ2eT375aJYTQH2j7vv/wokJOBgAwCMhkstWrV+fl5TU1NRUUFERGRgqFwqSkJLZCQEBAZWXlrl27VCpVaWnpsmXL2EdorFdffbWkpOT+/ft5eXm3b9/28/Oj5XyNu5TL5ba2tkVFRTrW9/DwSE5Obl+emJjo4uISGxublZXV2NhYUlIyf/78qqqqpKQk+gZTF4mJia6uru+++252drZSqayrq0tOTt6wYcO2bdvYJ2eRkZECgeDOnTs6HrMzaWlpGzZsKCsrU6vVZWVlq1atSk1N9fLyioqK4lajM3EEBAT08nQwaDAwSND1aPmOAoYuQkh6ejrfUQwIYWFhYWFh3d2L9kNnLViwQGOShQ8//JD57cjHwMBAuq9cLndycrp+/bpCoTAzMxOLxVOmTLlw4QL3+PX19VFRUQ4ODmKx2NfXNz8/n50LY9WqVbTOzZs3/fz8JBKJs7Pz7t272X39/PwsLS0vXrzYswbR5d748ssvyX/Wu+RavXq1kZFRRUUF/VhTU8NtAS8vr/aHWrx4MV3vkqu2tjY2NtbFxcXY2FgmkykUipycHLpJ93amk5yNHj3a2NjYxsYmICDg7Nmz3LNMnz5dKpW2trZqudLMzMz2X7UHDhxgKyiVypSUFIVCQSdCk0qlXl5eiYmJT5480ThUeHi4k5NTS0uLRrmhoeGECRO0xMDq2b0KfMF3/KCBnAz4hZyMpf/vOZqT6fOM3dKbnKy+vt7JySk6Orrfouszjx8/FovFUVFR+jldYWGhQCA4evRo+03IyV5UeHcJAAC8kclkmZmZx48f3717N9+xaMMwTExMjLm5eftZxPrD7du3Q0NDExIS5s2bp4fTwQCBnAwGkNOnT7u5ufV40JNUKuUOYt+2bVvfhtcbAzm2Pnf37t0//elPI0aMEAqF7CVv2rRJbwE8e/Zsx44dXl5eZmZmtra2M2fOzMzMZDqZEDU4OFjP4Q1lixcvFggEUqmUW+jp6VlQUJCdnd3Q0MBXYF168ODB7du3c3JydBzI2UvJycmbN2/evHkztzA+Pp7+b3r+/LkeYgD9Q072olGpVC+99BJ32sNBobS0NDg4OCEhQWPS7W5RqVQ///wzIWTOnDkMw8TFxfVdgL01kGPrWzU1NRMnTvzpp58yMjLq6+uZdr15+ltTU9P06dMPHTq0Y8eOhw8fFhQUSKXS4ODgX375pX3lw4cPd9j7Z+Cg61QWFRVVVFQIBII1a9bwHVEPRUZGsi9oVCqVxtZRo0ZlZWWZm5vzEpsu7O3tL1y44OHhoZ/Tbdmypf0Tsk8++YRtQ/0PyAA9QE72omEYpq2tTWO2SX2SSqXsQsi6W7t27aRJky5fvqwx6+Ng1LMWeGGkpKRUV1fv2LFj4sSJpqam/Xeiztp55cqVV65c+de//vWHP/xBLBaPGDHi0KFDdPInDZWVlbGxsQsXLuy/IHuPrlPJwvM8gBcY5ox90ZiZmZWWlvIdRbd9/vnnmKv6xXD16lVCyJgxY3g5+4MHD/bv3//f//3f3BkQJBIJXcNHw6JFi8LDw/38/Gj3cwAAfuE5GQwISMheGE+ePCGE8PW885///Ofz5891eU558ODBX3755cXu2AcAgwtyshfKyZMn2S7V9MEAt6SsrCwiIsLCwsLKyiooKIh9nEY7rAgEguHDh+fn5/v7+5uZmZmamk6bNo2d0nrTpk20Dvttd+bMGVpibW3NPU5TU1Nubi7dxMsSJRoGVwu0tramp6f/8Y9/tLe3F4vFY8aMSUpKom+i6+vruaME6Dus1tZWtiQsLIwepKamJiYmhk59ZGNjExoaSmee1GiN4uLiuXPnWllZ0Y+1tbW9auj/HPwf//gHIUQsFnPbqj06EZSrq6tQKLS0tJw5c+a5c+d0aQeitZ3p6oSWlpYrVqxwdnYWCoUjR46MiYnRWLKmvLx8xYoVBw8efAHelQPAi0MfE25AX9B9frI5c+YQQpqbmzVK5syZc/HiRZVKdfbsWbFYPH78eO5ecrlcIpH4+PjQOvn5+WPHjhUKhefPn2frSCSSyZMnc/fy8vLSmLyxfZ1ucXJyMjQ07HDTtGnThg0blpeXp2V3bj96roHQAp3FxkX7m3/88cd1dXU1NTV//etfDQwMuD2KFAqFgYHBr7/+yt3Lx8fnyJEj9N+VlZUjR460s7M7depUY2PjtWvXpkyZYmJiwp0OlLbGlClTzp0719TUdOnSJUNDw5qaGi2BMTrPT9b+9qN9/Ddu3MiWVFVVubi42NnZZWZmKpXK4uLi0NBQgUDAzqvZZTswnbQzPbu9vf2CBQtKS0sfP37897//XSKRuLm50QEHbDO+99579N/0xSU3vC5hzicNOt4boH+4VwcXPCcbQqKionx8fCQSyYwZMwIDA/Pz8zUejTQ1Ne3Zs4fW8fb2Tk1NbWlpWbZsGV8Ba2hra6N3bY+PMChaYOrUqQkJCZaWltbW1kuXLp0/f35SUhI7R8Dy5cvb2tq2b9/O1s/Nzb137154eDj9mJCQcPfu3e3bt8+aNUsqlXp4eBw7doxhmKVLl2qcaNWqVVOnTjU1NZ0wYUJrayv7tE8PEhIS7ty5s3PnzqCgIHNzczc3t7S0NAcHh5iYGHbgrfZ26Ax9PCwWiw8dOjR69GgLC4u33norISGhpKTks88+o3UOHDhw69atrVu39us1AgB0F3KyIYS7uq2zszMhpLKykltBIpGMGzeO/ThmzBhHR8eioqKqqiq9BanF+fPn6+rqfHx8enyEgd8CQUFB3Fd4hBC5XP7s2TN2HoeAgIAxY8YcOnTo0aNHtOTTTz9dunSpsbEx/Xjy5EkDAwPuZCj29vYeHh6XL1/WWOn597//fT9eiVZ0laHAwEC2RCQS+fv7Nzc3f/PNN0SHduiMRCIhhMyYMYP71nj27NmEEHrke/furVy58uDBg7Rmjx0/flwA/0EIiYiI4DsK6MDx48d7c5+DnvHf3Qf0RiaTsf8WCoWEEI0pMywsLDR2sbW1raysfPjwoYODgx4i7G8DvwWUSuVnn3124sSJ8vLy+vp6tpx2nKdiY2P//Oc/79mzZ+3atSUlJd99990XX3xBN6nVaqVSSX57paxbt24NHz6c/djLpKTHaJAmJiYafbnoSMnq6mqiWzt0aNSoUYQQKysrbiFdipsupEjflk6dOlVjx7Vr165du5YQcuvWrd/97nddXsXEiRM/+OCDLqsNEREREbGxsb35ewn6yY4dO/gOAboBORn8n0ePHjEMIxAI2JKHDx+S/3ylEUIMDAxaWlq4u3C/Lynu7oMO7y0we/bs//3f/01KSnrzzTetra0FAsHOnTs/+OAD7hvbBQsWrF69eteuXX/5y18+++yzt99+29LSkm4SiUQWFhYqlaq5uXkgDLDokEgkkslkSqWysbGRm5bRt5Z0knRd2qHDdvb19d2+fbvGc036Q6Q535IlS5YsWcLdmpqaunDhwo0bN3ZrOtbhw4fPnTtX9/ovtoiICB8fHzTIAPTVV1/xHQJ0A95dwv95+vRpfn4++/Hq1auVlZVyuZx9ROTg4FBRUcFWqK6uvnfvnsZBTE1N2azl5Zdf3r9/fz9H3Zf4agEjI6ObN28+f/48NzfX3t4+JibGxsaG5hzNzc0alUUi0Xvvvffw4cPPPvvsyJEjGt3dQkNDW1tb2eGi1JYtW0aMGNHa2tplJPoREhJCCDl16hRbolarc3JyxGKxQqHQsR06bOdZs2Y5OTmdOXOGOyEZHTHw+uuv9/NlAQD0CnIy+D8ymWz16tV5eXlNTU0FBQWRkZFCoTApKYmtEBAQUFlZuWvXLpVKVVpaumzZMvYBEuvVV18tKSm5f/9+Xl7e7du3/fz8+iq86dOnW1lZ9euKIvy2gKGh4dSpU6urqz/99NPa2trm5uZz587t27evfc333ntPLBavWbNmxowZGi/aEhMTXV1d33333ezsbKVSWVdXl5ycvGHDhm3btg2cJ2eJiYkuLi6xsbFZWVmNjY0lJSXz58+vqqpKSkqys7PTsR06bGeRSJSSkvLo0aN58+bdunWrvr7+yy+/TExMnDBhQkxMDB/XCgCgM95GfEI36TIXBu06zVqwYIHGUoMffvgh89txi4GBgXRfuVzu5OR0/fp1hUJhZmYmFounTJly4cIF7vHr6+ujoqIcHBzEYrGvr29+fr6Xlxc9zqpVq2idmzdv+vn5SSQSZ2fn3bt363h1Ha45yM6MQPn5+VlaWnLndNCg0UHq008/ZdottshXC3TZeevGjRsMw9TU1ERHRzs7OxsbG9vZ2b3zzjvx8fG0gpeXFzeSRYsWEUK+//779u1Ap/4aPXq0sbGxjY1NQEDA2bNn6ab2S0/q+ANidJjvoP3txzCMq6srt/D+/fu0cm1tbWxsrIuLi7GxsUwmUygUOTk57KF0aQctd9rFixcVCoVMJhMKhe7u7h999NGTJ0/aBxwdHa3RGgqFQpemwPwCGrq8N4AvuFcHFwHTi5kFQJ8yMjIiIiL67+c1bty42tpajaF5Q8rgaoEvvvhi9+7dBQUFejujQCBIT09HnyFCCJ15BD11WLg3Bizcq4ML3l0CDEr79u1bvnw531EA6Co1NZWdoEEqlWpsvXv3bnBwcENDQ21tLVvN09NTY6FS7laBQODt7a3HK+ie06dPu7m5ddhhgGGY3NzcJUuWuLm5iUQiW1tbX1/f1NRU7p/c8fHx9N0IV3x8PHvtEydO7N8LAD4gJwMYNFJSUkJCQlQq1b59+x4/fozHEjDo7N27l2EYlUrFLSwsLPT29g4ICDA3N7e2tmYYhg61KSwsjI2N5dakW/Py8ujiGfp8Tqy70tLS4ODghIQEdgJkDcXFxb6+viUlJcePH1cqlZcuXRoxYsTChQtXrlzJ1lm0aFFCQgKdn4X1ySef0DdchoaG/XsNwBPkZPD/rx5YVFRUUVEhEAi6NSOALrTMZ/jRRx/17bl6pr9boA+dPHnS0tJy7969x44dGzh99qFfSaVSXVZVH7DH166hoWH27NlvvPHG+++/zy0XiURWVlbJyclHjx7lK7aeWbt27aRJky5fvqxlNVUjI6OMjIyxY8eamJiMHj360KFDVlZWu3btUqvVtIKrq+uJEyc2b96ckZGhr8CBf/idDiQuLi4uLq7/jj/w+yz2dwv0laioqKioKL6jAOhLW7dura6uXrdunUa5iYnJkSNHZs2aFR0d7eXl5ebmxkt4PfD555+LxWItFdzd3Z89e8YtEQqFzs7OhYWFT58+FYlEtFAul4eFha1YsSI0NBR/gA0ReE4GAAD8YBgmJSVlwoQJjo6O7bcqFIo1a9Y0NjaGh4drdCwbyLQnZB2qr6+/deuWp6enxgocISEh5eXl3Jn84MWGnAwAoL/QeUlcXV2FQqGlpeXMmTPZdTw3bdpE3+Cz7w3PnDlDS9j14Olb9aamptzcXLqJPi+h5QKBYPjw4fn5+f7+/mZmZqamptOmTWOnC+7N8fWmqKjowYMHcrm8swrr168PCAi4cuXK0qVLtR9KS1OfPHmS7S9RVlYWERFhYWFhZWUVFBRUWlrKPUhNTU1MTMyoUaOEQqGNjU1oaGhhYWHvL1OLhoaG3Nzc4OBge3v7w4cPa2yly+/SpVphKEBOBgDQL6qrq8ePH5+WlpaUlFRbW/vDDz+Ympr6+/unpKQQQtasWcP8duK61157jWEYdsY7QkhcXBytM3nyZNq5my7GQMvlcnl9ff2yZcs2bdpUXV3973//u66ubvr06d9//30vj0/pYZbma9euEUK4y7BqMDAwOHLkiLOzc0pKypEjRzqrpr2pX3/9dYZh5syZQwiJjY2NjY2tqKhIT0//7rvv3nzzTfYgVVVV48ePz8jI2LNnT11d3fnz5+vq6nx8fNrP6tdXNm3aJJPJfH19DQ0NT5w48corr2hUcHJyIv9pJRgKkJMBAPSLhISEO3fu7Ny5MygoyNzc3M3NLS0tzcHBISYmprMRed3V1NS0Z88eHx8fiUTi7e2dmpra0tKisdxWj7W1tdFErU+O1iG6MqnGCzsN1tbWGRkZxsbG0dHRN2/e7LCO7k0dFRVFm2vGjBmBgYH5+fm1tbXsQe7evbt9+/ZZs2ZJpVIPD49jx44xDNPlI7oeW7NmjVqtvnHjhru7u6en58aNGzUqmJubCwQCjfVb4QWGnAwAoF/QhQ0CAwPZEpFI5O/v39zc3FdvoyQSCX29RY0ZM8bR0bGoqKhPvsXZB0W9P1RnaC8xY2Nj7dUmTpy4bdu2pqam8PDw9iufku409fjx49l/Ozs7E0IqKyvpx5MnTxoYGAQFBbEV7O3tPTw8Ll++3H9TSdOlJvbu3RscHLxu3bpvv/1Wo4KRkVGHlwwvJORkAAB9T61WK5VKExMTjQkR7OzsCCHV1dV9chYLCwuNEroA68OHD/vk+P3NxMSEEKIxCLFDMTExERER165d05gyg3SzqbnP5IRCISGkra2NPUhbW5tMJuPO1/PTTz8RQm7dutWzC9Td7NmzCSFZWVka5a2trT0YNACDFIbXAgD0PZFIJJPJlEplY2MjN1egr9Ls7e3pRwMDg5aWFu6O9fX1GocSCASdneXRo0cMw3Ar0GyMZma9P35/c3BwIIQolUpdKqekpBQWFh48eJBmciwdm1o7kUhkYWGhUqmam5t5mXiCToFRV1fHLWxoaGAYhrYSDAV4TgYA0C9CQkIIIdyJDNRqdU5OjlgsVigUtMTBwaGiooKtUF1dfe/ePY3jmJqasnnVyy+/vH//fnbT06dP6ZT31NWrVysrK+VyOfst3svj9zfaq13HN4NSqfTrr7+WSCR79uzR2KRLU3cpNDS0tbWVHbhKbdmyZcSIEdyhD70XFxcXGRmpUZidnU1++2qVEEJ/du37/sOLCjkZAEC/SExMdHFxiY2NzcrKamxsLCkpmT9/flVVVVJSEn2tRggJCAiorKzctWuXSqUqLS1dtmwZ+4iL9eqrr5aUlNy/fz8vL+/27dt+fn7sJplMtnr16ry8vKampoKCgsjISKFQmJSUxFbozfH1MO5SLpfb2toWFRXpWN/DwyM5Obl9uS5N3aXExERXV9d33303OztbqVTW1dUlJydv2LBh27Zt7JOzyMhIgUBw584dHY/ZmbS0tA0bNpSVlanV6rKyslWrVqWmpnp5eWlMCk1n4ggICOjl6WDQYGCQoOvR8h0FDF2EkPT0dL6jGBDCwsLCwsJ0qVlbWxsbG+vi4mJsbCyTyRQKRU5ODrdCfX19VFSUg4ODWCz29fXNz89n56pYtWoVrXPz5k0/Pz+JROLs7Lx79252X7lc7uTkdP36dYVCYWZmJhaLp0yZcuHChb46vp+fn6Wl5cWLF7u8TF3ujS+//JL8Z71LrtWrVxsZGVVUVNCPNTU13G8oLy+v9odavHgxXe+SS0tTa0xm8eGHHzK/HUwaGBhIa9JJzkaPHm1sbGxjYxMQEHD27FnuWaZPny6VSltbW7VcaWZmZvuv2gMHDrAVlEplSkqKQqGgE6FJpVIvL6/ExMQnT55oHCo8PNzJyamlpUWj3NDQcMKECVpiYOl+r8JAgO/4QQM5GfALORlrgHzP0ZyM7ygYpnc5WX19vZOTU3R0dL9F12ceP34sFoujoqL0c7rCwkKBQHD06NH2m5CTvajw7hIAAHgjk8kyMzOPHz++e/duvmPRhmGYmJgYc3Pz9rOI9Yfbt2+HhoYmJCTMmzdPD6eDAQI5GQAA6MnixYsFAoFUKuUWenp6FhQUZGdnNzQ08BVYlx48eHD79u2cnBwdB3L2UnJy8ubNmzdv3swtjI+PpzN0PH/+XA8xgP4hJwMAGGToOpVFRUUVFRUCgWDNmjV8R9S1yMhI9gWNSqXS2Dpq1KisrCxzc3NeYtOFvb39hQsXPDw89HO6LVu2tH9C9sknn7Bt2K9jL4AvmJ8MAGCQiYuLi4uL4zsKAOhjeE4GAAAAwD/kZAAAAAD8Q04GAAAAwD/kZAAAAAD8Qx//QSYjI4PvEGDo0pgPfcii6zPiPyMX7o2Bqby8fPjw4XxHAboSML9dYgIGrIyMjIiICL6jAACAwSQsLOyrr77iOwrQCXIyABj0BAJBenr63Llz+Q4EAKDn0J8MAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAF8GxgYAACAASURBVAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH/IyQAAAAD4h5wMAAAAgH8ChmH4jgEAoHuio6OLi4vZjz/99JOLi4ulpSX9aGho+Pe//3348OE8RQcA0BNGfAcAANBtdnZ2+/fv55ZcuXKF/ffo0aORkAHAoIN3lwAw+MyfP7+zTUKh8J133tFjLAAAfQPvLgFgUHrllVeuX7/e4W+w4uJiNzc3/YcEANAbeE4GAIPSW2+9ZWhoqFEoEAjkcjkSMgAYjJCTAcCg9Oabbz5//lyj0NDQ8O233+YlHgCAXsK7SwAYrCZNmvTDDz+0tbWxJQKB4P79+05OTjxGBQDQM3hOBgCD1cKFCwUCAfvRwMDA19cXCRkADFLIyQBgsAoPD+d+FAgEb731Fl/BAAD0EnIyABisrK2t/f392Z7+AoEgJCSE35AAAHoMORkADGKRkZG0U6yhoaFCobCysuI7IgCAHkJOBgCDWGhoqFAoJIQwDBMZGcl3OAAAPYecDAAGMYlEEhQURAgRCoWzZ8/mOxwAgJ5DTgYAg9uCBQsIISEhIRKJhO9YAAB6geFIT0/nOxwAAACAISEsLIybhhm1r4HMDAAGl9TU1Hnz5hkZdfALDTTk5eXt3LkTv+epiIiI2NhYHx8fvgMZEHBv6NmOHTs0Sn4zj39GRkZERARm9geAweXp06cmJiZ8RzE44Pc8l0AgSE9Pnzt3Lt+BDAi4N/SMzrD41VdfsSXoTwYAgx4SMgB4ASAnAwAAAOAfcjIAAAAA/iEnAwAAAOAfcjIAAADotrt37wYHBzc0NNTW1gr+w9PT8+nTp9xq3K0CgcDb25uvgLt0+vRpNze3DkdwMwyTm5u7ZMkSNzc3kUhka2vr6+ubmprKHRIRHx/fy1GryMkAAAC6QaVSvfTSS3QBiSGrsLDQ29s7ICDA3Nzc2tqaYZj8/HxaHhsby61Jt+bl5VlZWTEMU1BQwFPI2pSWlgYHByckJDx48KDDCsXFxb6+viUlJcePH1cqlZcuXRoxYsTChQtXrlzJ1lm0aFFCQsLatWt7HAZyMgAAgG5gGKatra2trY2vAKRSqa+vL19nJ4Q0NDTMnj37jTfeeP/997nlIpHIysoqOTn56NGjfMXWM2vXrp00adLly5fNzMw6q2NkZJSRkTF27FgTE5PRo0cfOnTIyspq165darWaVnB1dT1x4sTmzZszMjJ6FgZyMgAAgG4wMzMrLS09ffo034HwZuvWrdXV1evWrdMoNzExOXLkiIGBQXR0dElJCS+x9cznn38eHx+vZd5pd3f3Z8+eWVpasiVCodDZ2VmtVnPf1crl8rCwsBUrVrS2tvYgDORkAAAAoCuGYVJSUiZMmODo6Nh+q0KhWLNmTWNjY3h4uEbHsoFMLBZ3d5f6+vpbt255enrKZDJueUhISHl5+alTp3oQBnIyAAAAXZ08eZLtrk5zDm5JWVlZRESEhYWFlZVVUFBQaWkp3Wvbtm20wvDhw/Pz8/39/c3MzExNTadNm5abm0vrbNq0idZh30ueOXOGllhbW3OP09TUlJubSzfpf0mxoqKiBw8eyOXyziqsX78+ICDgypUrS5cu1X6oR48eLV++3NXVVSgUWlpazpw589y5c3STLq1K1dTUxMTEjBo1SigU2tjYhIaGFhYW9v4ytWhoaMjNzQ0ODra3tz98+LDG1nHjxhFCvvnmm54cuv0a5AwAALyg8HueixCSnp7egx3nzJlDCGlubtYomTNnzsWLF1Uq1dmzZ8Vi8fjx47l7yeVyiUTi4+ND6+Tn548dO1YoFJ4/f56tI5FIJk+ezN3Ly8uL9o7XUoeaNm3asGHD8vLyenBFjM73xpdffkkI+fjjjzXK8/PzZTIZ/XdNTY2zszMhhI5MZDh9/FlVVVUuLi52dnaZmZlKpbK4uDg0NFQgEBw4cICt02WrVlZWjhw50s7O7tSpU42NjdeuXZsyZYqJicnFixd71ghOTk6GhoZaKmzcuJGmT1OnTr1y5Ur7CkqlkhDi5+fX5bnCwsI01iDHczIAAIC+ERUV5ePjI5FIZsyYERgYmJ+fX1tby63Q1NS0Z88eWsfb2zs1NbWlpWXZsmV9cva2tjb61d4nR+tMVVUVIUTjhZ0Ga2vrjIwMY2Pj6OjomzdvdlgnISHhzp07O3fuDAoKMjc3d3NzS0tLc3BwiImJ0Rj8qKVVExIS7t69u3379lmzZkmlUg8Pj2PHjjEM0+Ujuh5bs2aNWq2+ceOGu7u7p6cnm6KxzM3NBQIBbaXuQk4GAADQN8aPH8/+mz4oqqys5FaQSCT03RY1ZswYR0fHoqKinn2Fazh//nxdXZ2Pj0/vD6UFfWNrbGysvdrEiRO3bdvW1NQUHh7e3NzcvsKJEycIIYGBgWyJSCTy9/dvbm7WePGnpVVPnjxpYGDAnZfE3t7ew8Pj8uXL5eXl3b00HQmFQnd397179wYHB69bt+7bb7/VqGBkZNThJXcJORkAAEDf4D49EgqFhBCNKTMsLCw0drG1tSWEPHz4sP+j6xsmJiaEkGfPnnVZMyYmJiIi4tq1axpTZhBC1Gq1Uqk0MTHRmHvCzs6OEFJdXc0t7KxV6UHa2tpkMhl3WtqffvqJEHLr1q2eXaDuZs+eTQjJysrSKG9tbe3BoAFCiL77BgIAAAxZjx49YhhGIBCwJTQbo5kZIcTAwKClpYW7S319vcZBuLvrn4ODAyGE9prqUkpKSmFh4cGDB2kmxxKJRDKZTKlUNjY2ctMy+tbS3t5el4OLRCILCwuVStXc3Kz/sQ40AEJIXV0dt7ChoYFhGNpK3YXnZAAAAHry9OlTOt89dfXq1crKSrlczn6FOzg4VFRUsBWqq6vv3buncRBTU1M2b3v55Zf379/fz1H/xiuvvEII0fHNoFQq/frrryUSyZ49ezQ2hYSEEEK4c0ao1eqcnByxWKxQKHQMJjQ0tLW1lR27Sm3ZsmXEiBE9myGsM3FxcZGRkRqF2dnZ5LevVgkh9MdHW6m7kJMBAADoiUwmW716dV5eXlNTU0FBQWRkpFAoTEpKYisEBARUVlbu2rVLpVKVlpYuW7aMfYTGevXVV0tKSu7fv5+Xl3f79m0/Pz9aPn36dCsrq0uXLvXrJcjlcltb26KiIh3re3h4JCcnty9PTEx0cXGJjY3NyspqbGwsKSmZP39+VVVVUlISfYOpi8TERFdX13fffTc7O1upVNbV1SUnJ2/YsGHbtm3sk7PIyEiBQHDnzh0dj9mZtLS0DRs2lJWVqdXqsrKyVatWpaamenl5RUVFcavRmTgCAgJ6cg7uIEyMkQYAeLHh9zwX6f5cGLRnOmvBggV5eXnckg8//JD57cjHwMBAuq9cLndycrp+/bpCoTAzMxOLxVOmTLlw4QL3+PX19VFRUQ4ODmKx2NfXNz8/38vLix5n1apVtM7Nmzf9/PwkEomzs/Pu3bvZff38/CwtLXs8DYTu98bq1auNjIwqKirox5qaGu71enl5td9l8eLFGnNhMAxTW1sbGxvr4uJibGwsk8kUCkVOTg7dpHur0knORo8ebWxsbGNjExAQcPbsWe5Zpk+fLpVKW1tbtVxRZmZm+wSJOyuHUqlMSUlRKBR0IjSpVOrl5ZWYmPjkyRONQ4WHhzs5ObW0tHTZjO3nwhBwLzIjIyMiIoLp52G0AADAF/ye5xIIBOnp6XPnztXP6caNG1dbW9t/4wF7Sfd7Q6lUenh4BAUF7du3Tw+B9UZ9fb2jo+OCBQsOHDigh9MVFRV5enqmpaXNmzevy8rh4eGEkK+++ootGUDvLrnTHPMdC/Ds2LFj9GbQ6BbKF5VKxR3Uo/EHHNfKlSvZaps2bdLl4H115+fn57/zzjsuLi5isXjYsGGvvPLKG2+8sXfvXo05r19UUqlUoFVBQQGP4Q20W1pHd+/e/dOf/jRixAihUNjdG7uXGIbJzc1dsmSJm5ubSCSytbX19fWlE5CydR4/frxv377p06cPGzZMLBa/9NJLCxYs0P2dGvSYTCbLzMw8fvz47t27+Y5FG4ZhYmJizM3N288i1h9u374dGhqakJCgS0LWoQGUk8XFxTEMo2W5Bi6VSvXSSy9xpySBF8m8efMYhvH39+cW8vhDl0qlDMP8/PPP9GNn/70fPXpE/2pcsGABwzBr1qzR5eC9v/Pb2tpWrlw5adIkW1vb7Ozs+vr6Gzdu7Nixo6Gh4b333vvd737Xt31dByaVSkV/QHPmzGn/jkD7/JZ6MNBuaV3U1NRMnDjxp59+ysjIqK+vZ9q9TupXxcXFvr6+JSUlx48fVyqVly5dGjFixMKFC1euXMnWWbly5dKlS+fMmXP9+vVHjx4dPHiwsLDQy8vr5MmTeotzyPL09CwoKMjOzm5oaOA7lk49ePDg9u3bOTk5Og7k7KXk5OTNmzdv3ry5x0cYQDlZZ6RSKbv4F4thmLa2No15X3iJBPSGlx+6BrFYPHLkyOzs7A4fuuzYsYPOZ9gndL/z165du23btj179mzdutXd3V0kEtnZ2f3xj388c+bMzJkz++S80B8Gwi2tRUpKSnV19Y4dOyZOnGhqatp/J+rsljMyMsrIyBg7dqyJicno0aMPHTpkZWW1a9cutVrN1nn33XeXLVtmb29vamrq5+eXlpb2/Pnzv/zlL/0Xbc/Qx+FFRUUVFRUCgUDHP9gGuFGjRmVlZZmbm/MdSKfs7e0vXLjg4eGhn9Nt2bKlx0/IqME6P5mZmdkQeSMDrIHwQzcwMIiPj1+8ePGmTZs0/havr6/fu3fv9u3b33nnnf4LoH0j3Lx585NPPvHy8lq0aJFGZUNDw7Vr19LR2kNc+xmeBoKBcEtrcfXqVULImDFjeDm7u7u7xqykQqHQ2dm5sLDw6dOndF6olJQUjb3kcrlYLC4tLWV+OwcY7+Li4uLi4viOAga6QfCcDGBA+dOf/uTk5PTPf/7zypUr3PK//vWvs2bNcnV11XM8+/fvb2tro31F2/Px8WEYhpfZFAcIX1/fQ4cO8R3FoPTkyRNCiMY06zyqr6+/deuWp6enljfRTU1Nzc3Nr7zyyoBKyAB01O2c7OTJk2xPz7t370ZERJiZmVlZWS1cuPDx48dlZWWzZ882MzNzcHBYtGhRY2Mj3WvTpk10F/YB9ZkzZ2iJtbV1Z+eiD3ubmppyc3NpZfrVwo2BLrylCzpc1tXVVSQSDR8+fMaMGYcOHWJXpGptbU1PT//jH/9ob28vFovHjBmTlJTEvlPoLBKqpqYmJiaGjo+1sbEJDQ2l05Owbt68+frrr8tkMlNT09///vdZWVkzZsygx2HnNWHDEwqFlpaWM2fOPHfuXPs2Ly4unjt3rpWVVfsuzLTjbWtrK1sSFhamS8toiZ976rKysoiICAsLCysrq6CgII2/77U3r/YL1GgliUTi5+d34cIF7tb2P3TdY9Ol/XUnEolWrlzJMAy304BKpfrb3/62evVqjcp6uPP//e9/E0LGjh2rS/BD+T5vfxbc0tpb6R//+AchRCwWc2/g9rS3Q29uOVZDQ0Nubm5wcLC9vf3hw4e1RE6HsH344Yc9u3AAnnG7weo+N8mcOXMIIaGhoQUFBSqViv4nmTlz5pw5c37++efGxkba0/mDDz7g7iWRSCZPnswt8fLy0piwhE7fon0vbgzNzc26BFxVVeXi4mJvb5+ZmdnQ0FBdXU27ae/YsYNWoHOTfPzxx3V1dTU1NX/9618NDAxo52vtkVRWVo4cOdLOzu7UqVONjY3Xrl2bMmWKiYkJO0PMrVu3LCwsnJyc/vWvf9EKM2bMsLGxEYlEGuHZ2dllZmYqlcri4uLQ0FCBQMCdHIVe75QpU86dO9fU1HTp0iVDQ8OamhqFQmFgYPDrr79yo/Lx8Tly5IguLdNl/Oyp58yZc/HiRZVKdfbsWbFYPH78eN2bt8sL1GilK1euBAQEjBo1ittKTEc/9C5j06X9GYaZNm3asGHD8vLytLTVzz//LJFIGIZ58uSJnZ2dgYHB9evX6aZPPvlk7ty5DMP87//+L/lPH39Wv975dPrvH374QUvkrBf7PmcHYWj44osvOmzDF/uW7kx3f89zY6N9/Ddu3KjRUFraoce3HIsdUjN16tQrV65oCbi6utrOzi4qKkqXq6NI9+cne4Fh7jo9az8/Wa9yslOnTrEltA/d999/z5a4uLi8/PLL3L14zMloFx+N/3uvvfYaNyebOnUqd2tkZKSxsbFSqdQeydtvv00I4X4xVFVViUQidtI8+lLp+PHjbIWHDx+amppyf4HS8I4ePcqWPH361NHRUSwWV1dXc6/39OnTGgF88803hJD33nuPLblw4YKOE9bpEj976szMTLaEPpmoqanhxq+lebu8wPatVFFRIRKJdPwC0xKbLu3PMMyUKVO6nGuRzckYhtmyZQshJDIykmGYpqYmOzu7oqIihr+c7Mcff9QSOevFvs87HHc5efLkznKyF/uW7kzf5mRdtkOPbzkutVp948aN//f//p+hoeGGDRs6rFNbWztu3LiIiAjtU4NqQE7GhZxMz9rnZL3qZeLt7c3+29HR8ZdffuGWODk5DZx5YujMyxpj0Lh9n4OCgjRGpMvl8tTU1F9++cXHx0fLkU+ePGlgYMDd197e3sPD4/Lly+Xl5cOHDz9z5gwhhLt6l42Njbu7+y+//KIRXmBgIFsiEon8/f2//PLLb7755q233mLLf//732sEEBAQMGbMmEOHDm3YsMHKyooQ8umnny5dutTY2FhL2LrHz5Zzl/SiowsrKyvpC7gum7fLC2zfSo6Ojm5ubiUlJbpchZbYdGl/Qsj58+d1ORHrvffe27p169GjR9evX5+ZmTlx4kQd3x72OUdHx6qqqtraWl0qD9n7vEMv9i2tH122Q49vOS6hUOju7r53794HDx6sW7fOx8dnxowZ3ApNTU0KheK//uu/Dh8+bGho2K1L0OcEHwMcbYqMjAy+AxkqNL5kSS/HXXJHwBoYGBgaGnLHSxsaGg6QMd5qtVqpVJqYmGjprKpUKj/77LMTJ06Ul5dzh2jRXq7aj0wI6bDP6a1bt2xsbBobG01MTKRSKXeTpaVll+HRBb+qq6u5hRKJpP2JYmNj//znP+/Zs2ft2rUlJSXffffdF198oSVs3ePn3i7cOkKhkBBCf75dNm+XF6hWqztsJVtbWx2/wLTE1mX794xUKo2NjV27du369evPnz9Pe97wYsqUKZcvX75y5You014MwftcoxcXF27pXtLlZ9qzW64zs2fPPnHiBO0/xxa2trbS1Wz+/ve/dzchI4Ts3Llz586dPQjmRRUREcF3CEOIRndYvY67NDAwYJeyp3QZoC7o9fAZkUgkk8mePn3Kjjlob/bs2Rs3bly0aFFJSUlbWxvDMDt27CCEMJw5o9tHIhKJLCwsjIyMnj171v6x5LRp00QikZmZ2dOnT1UqFXfHhw8fdhnegwcPCCG6zHS3YMECOzs7Om3PZ5999vbbb+v4C7rL+HU8iPbm7fICO2uluro6XQLQHluX7d9jS5culclkaWlpcrmc+4S4vX6986Ojo42MjI4fP97h1r/85S8GBgY3b96kH4fmfd5dQ/aW7kEwXf5Me3bLaTkjadeM0dHRarU6IyODHR/wu9/9Tvd1uPHukoV3l3rWfnySXnMyBweHiooK9mN1dfW9e/e63MvU1JT9Pnv55Zf379/fg1OHhIQQQk6fPs0t9PT0/OCDDwghz58/z83Ntbe3j4mJsbGxob8guAOstEQSGhra2tqam5vLrblly5YRI0bQydPp0wv6uoGqrq7W+FuZhnfq1Cm2RK1W5+TkiMVi7huKzohEovfee+/hw4efffbZkSNHli1b1uUurC7j14X25iU6XGD7VqqtrS0uLtb9QjqjS/v3jEwmW758uUwm63L6x369893c3NavX19QUHDw4EGNTcXFxcnJyXPnznV3dydD+z739vY+duyY7vWH5i3dA9rboTe3XFxcXGRkpEZN+vqY+2L3o48++uWXX/7xj3/QdA1gcOOmbL3p+6lQKAwNDbl1pkyZwnaIpt5//31CyN/+9rfGxsZff/117ty5Tk5OXfZ0fu2112Qy2b179y5evGhkZMSOdOvBuEsHB4esrKyGhob79+8vXrzYzs7u7t27tML06dMJIVu3bq2pqXny5Ml33303YsQIQgh3efkOI3nw4IGrq+vo0aNPnz5dX19PV9cxNTVl//b69ddfhw0bxg6Sunr16muvvTZy5MjOxqM1NDSwY5f279+vpc25ampq6JD1DheW0aLL+Ds89apVqwghP//8s47N2+UFarTSL7/8olAobG1tdewQrSU2Xdqf6ea4Sy067OOvhzs/Pj7e2Nh41apVxcXFarW6vLw8JSXFwcHB19dXpVKx1V7s+1zL2kpeXl7cfuhD4ZbuTP+Nu+ywHXp8y61YsUIgEPzP//zPnTt3nj59eufOHTo7v5eX15MnT+iOWl5ea/+/zCJ4TsaB52R61gfjLjW6Q3744Yf5+fncksTERPq1xFq/fj3dt76+PioqysHBQSwW+/r65ufne3l50TqrVq369NNPNY5M97p586afn59EInF2dt69ezfDMLRXKUvj+68ztbW1sbGxLi4uxsbGDg4O8+bNKykpYbfW1NRER0c7OzsbGxvb2dm988478fHx9PjsyLL2kVB0ep7Ro0cbGxvb2NgEBARwf90wDFNcXPz666+bm5ubmppOmjTp+++/nzp1qqmpaWfhyWQyhUKRk5PTYZt39jOi07hzh77qSEv87X/cDOeNAyEkMDBQl+bVfoEarURH/mdlZbGLA/75z39u/0PXPTZd2t/Pz0/7uEtuDyeFQtFhHY0f09/+9jdarp87/8cff1y4cCG9h83MzCZOnJiUlKRWq7l1XuD7vMMuaFw0Jxs6t3RndPk93+GdpjEf8v3793Vphx7fckqlMiUlRaFQ0CnxpFKpl5dXYmIim5AxDMMdW6ABOVkPICfTs/Y5mYDh/J/PyMiIiIhg2n21QJ9zd3dvbm6+e/duHx7ziy++2L17d4frMIKG/mh/aA/3ud7o3tT4Pc8lEAjS09Pnzp3LdyADAu4NPaPz2tCJjimsrdTvqqurhw0bxl24raysrLS0lD7S70P79u1bvnx53x7zBaC39h/icJ/rDW5pgBcVcjJ9ePz4cXR09P379588efLjjz9GRESYm5uvXbu290dOSUkJCQlRqVT79u17/Pgx/trrUP+1P3DhPtcb3NIwENy9ezc4OLihoaG2tpZdFszT01NjzUPuVoFAoH2UOr9Onz7t5ubW4QJfDMPk5uYuWbLEzc1NJBLZ2tr6+vqmpqZyHyvGx8fT97899oLkZO0XxWN99NFH/MZmb2//7bff1tfX/+EPf7C0tAwODn7ppZd+/PHH0aNH98nxT548aWlpuXfv3mPHjrW/kwZyy+hHf7c/UPze50MKbmkYCAoLC729vQMCAszNza2trRmGoZ3LCwsLY2NjuTXp1ry8PDqwaWB2PCgtLQ0ODk5ISKAzubRXXFzs6+tbUlJy/PhxpVJ56dKlESNGLFy4cOXKlWydRYsWJSQk9OqvI27nMvTvAwB4seH3PBfRYx//LpeQ4v34ut8bSqVy+PDh0dHR3ML8/HyRSESX2UhLS9PYhc3JBqY333wzMTHx2bNnTk5OGpNIUDdu3DAyMqqrq2NL1Gq1lZWVSCR6+vQpW1hYWEg7Kepy0vZ9/F+Q52QAAACgH1u3bq2url63bp1GuYmJyZEjRwwMDKKjo/maM69nPv/88/j4eC3P4N3d3Z89e8adp1ooFDo7O6vVau67WrlcHhYWtmLFCt0n+ORCTgYAAAC6YhgmJSVlwoQJjo6O7bcqFIo1a9Y0NjaGh4drdCwbyMRicXd3qa+vv3Xrlqenp8aicyEhIeXl5dyJlHWHnAwAAEAbOjmfq6urUCi0tLScOXPmuXPn6KZNmzbRHrq+vr605MyZM7SErhlPCNm2bZtAIGhqasrNzaWb6PMYWi4QCIYPH56fn+/v729mZmZqajpt2jR2zYzeHL+fFBUVPXjwQC6Xd1Zh/fr1AQEBV65cWbp0qfZDaWnYkydPsr2fy8rKIiIiLCwsrKysgoKCSktLuQepqamJiYmh89jZ2NiEhoYWFhb2/jK1aGhoyM3NDQ4Otre3P3z4sMbWcePGEUK++eabnhya+yIT/QwAAF5s+D3PRXToT8Zdq0CpVLJrFRw4cICt074vl5eXl0b3qc76e8nlcolE4uPjc/HiRZVKlZ+fP3bsWKFQeP78+T45vi6LlFA63htffvklIeTjjz/WKM/Pz5fJZPTfNTU1zs7OhBA6MpHpqD+ZLg1LF5OYM2cObZyzZ8/SGZjZCpWVlSNHjrSzszt16lRjY+O1a9emTJliYmKiZQJw7TrrT8bauHEjTZ+mTp165cqV9hWUSiUhxM/Pr8tzoT8ZAABANyQkJNy5c2fnzp1BQUHm5uZubm5paWkODg4xMTGdjdHrrqampj179vj4+EgkEm9v79TU1JaWlm6t6KoFXfqd6buZYKuqqgghGi/sNFhbW2dkZBgbG0dHR9+8ebPDOro3bFRUFG2cGTNmBAYG5ufn19bWsge5e/fu9u3bZ82aJZVKPTw8jh07xjBMl4/oemzNmjVqtfrGjRvu7u6enp5sisYyNzcXCAS0lboLORkAAECn6EpT3HWcRCKRv79/c3NzD99PtSORSOgLL2rMmDGOjo5FRUU9+17XcP78+bq6Oh8fn94fiqK9F6JY3AAAIABJREFUxIyNjbVXmzhx4rZt25qamsLDw9svPE+607DcVefp47fKykr68eTJkwYGBkFBQWwFe3t7Dw+Py5cvl5eXd/fSdCQUCt3d3ffu3RscHLxu3bpvv/1Wo4KRkVGHl9wl5GQAAAAdU6vVSqXSxMTEzMyMW25nZ0cIqa6u7pOzWFhYaJTY2toSQh4+fNgnx+9bJiYmhBDuShKdiYmJiYiIuHbt2vvvv6+xqVsNy30mJxQKCSFtbW3sQdra2mQyGXf2zZ9++okQcuvWrZ5doO5mz55NCMnKytIob21t7cGgAULIkJ56EQAAQAuRSCSTyZRKZWNjIzd7oC/X7O3t6UcDA4OWlhbujvX19RqHEggEnZ3l0aNHDMNwK9BsjGZmvT9+33JwcCCE0F5TXUpJSSksLDx48CDN5Fg6Nqx2IpHIwsJCpVI1NzfzMpW0SCQihNTV1XELGxoaGIahrdRdeE4GAADQqZCQEEIId2oDtVqdk5MjFosVCgUtcXBwqKioYCtUV1ffu3dP4zimpqZsXvXyyy/v37+f3fT06VM6CT519erVyspKuVzOfq/38vh965VXXiGE6PhmUCqVfv311xKJZM+ePRqbdGnYLoWGhra2trLDVKktW7aMGDGiZzOEdSYuLi4yMlKjMDs7m/z21SohhP6kaCt1F3IyAACATiUmJrq4uMTGxmZlZTU2NpaUlMyfP7+qqiopKYm+aCOEBAQEVFZW7tq1S6VSlZaWLlu2jH3ExXr11VdLSkru37+fl5d3+/ZtPz8/dpNMJlu9enVeXl5TU1NBQUFkZKRQKExKSmIr9Ob406dPt7KyunTpUl81iFwut7W1LSoq0rG+h4dHcnJy+3JdGrZLiYmJrq6u7777bnZ2tlKprKurS05O3rBhw7Zt29gnZ5GRkQKB4M6dOzoeszNpaWkbNmwoKytTq9VlZWWrVq1KTU318vKKioriVqMzcQQEBPTkHNxBmBgjDQDwYsPveS6i29pKtbW1sbGxLi4uxsbGMplMoVDk5ORwK9TX10dFRTk4OIjFYl9f3/z8fC8vL/olu2rVKlrn5s2bfn5+EonE2dl59+7d7L5yudzJyen69esKhcLMzEwsFk+ZMuXChQt9dXw/Pz9LS0td5obQ/d5YvXq1kZFRRUUF/VhTU8PNK7y8vNrvsnjx4vZrK2lp2Ly8PO4xP/zwQ+a3Q0cDAwNpTTrJ2ejRo42NjW1sbAICAs6ePcs9y/Tp06VSaWtrq5YryszMbJ8gcWflUCqVKSkpCoWCToQmlUq9vLwSExOfPHmicajw8HAnJ6eWlpYum7H9XBgC7kVmZGREREQwfTdiFgAABhT8nueiSxPOnTuXxxjGjRtXW1vbf4MEdaf7vaFUKj08PIKCgvbt26eHwHqjvr7e0dFxwYIFBw4c0MPpioqKPD0909LS5s2b12Xl8PBwQshXX33FluDdJQAAAHSDTCbLzMw8fvz47t27+Y5FG4ZhYmJizM3N288i1h9u374dGhqakJCgS0LWIeRkAAAA0D2enp4FBQXZ2dkNDQ18x9KpBw8e3L59OycnR8eBnL2UnJy8efPmzZs39/gIyMkAAAB4QNepLCoqqqioEAgEa9as4Tui7hk1alRWVpa5uTnfgXTK3t7+woULHh4e+jndli1bevyEjML8ZAAAADyIi4uLi4vjOwoYQPCcDAAAAIB/yMkAAAAA+IecDAAAAIB/yMkAAAAA+NdBH386iRkAALx46Nyk+D3P2rFjB3fSzqEM94aeXbp0aeLEidyS38zjn5eXt337dr1HBQDQK9nZ2Z6envqZgggAoK/4+PgsX76c/SjAChsAMNgNhBVyAAB6Cf3JAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf8jJAAAAAPiHnAwAAACAf0Z8BwAA0G319fUMw3BLmpqaHj9+zH6USqXGxsZ6jwsAoOcEGr/XAAAGvunTp587d66zrYaGhhUVFXZ2dvoMCQCgl/DuEgAGnzfffFMgEHS4ycDA4A9/+AMSMgAYdJCTAcDgExYWZmTUcdcLgUDw1ltv6TkeAIDeQ04GAIOPpaVlQECAoaFh+00GBgYhISH6DwkAoJeQkwHAoBQZGdnW1qZRaGRkFBgYKJPJeAkJAKA3kJMBwKAUHBwsEok0Cp8/fx4ZGclLPAAAvYScDAAGJVNT05CQEI0JL8Ri8axZs/gKCQCgN5CTAcBgNX/+/GfPnrEfjY2Nw8LCxGIxjyEBAPQYcjIAGKwUCgW369izZ8/mz5/PYzwAAL2BnAwABitjY+N58+YJhUL60cLCwt/fn9+QAAB6DDkZAAxib775ZktLCyHE2Ng4MjKys0nLAAAGPqytBACDWFtbm6Oj44MHDwghFy5cmDx5Mt8RAQD0EJ6TAcAgZmBgsHDhQkKIg4PDpEmT+A4HAKDn8Jy/VzIyMvgOAWCos7a2JoRMmDDhq6++4jsWgKFu0qRJw4cP5zuKwQrvLnuls1WQAQAAhqD09PS5c+fyHcVghedkvYX7D4Bf4eHh5eXleXl5fAfCv4yMjIiICPylzQoPDyeE4AGq3uA5RS+hPxkADHp4VwIALwDkZAAAAAD8Q04GAAAAwD/kZAAAAAD8Q04GAAAAwD/kZAAAAIQQcvfu3eDg4IaGhtraWsF/eHp6Pn36lFuNu1UgEHh7e/MVcJdOnz7t5ubW4ZpjDMPk5uYuWbLEzc1NJBLZ2tr6+vqmpqZyx+3Gx8enp6frMd6hDjkZAMBQp1KpXnrppaCgIL4D4VNhYaG3t3dAQIC5ubm1tTXDMPn5+bQ8NjaWW5NuzcvLs7KyYhimoKCAp5C1KS0tDQ4OTkhIoCuPtVdcXOzr61tSUnL8+HGlUnnp0qURI0YsXLhw5cqVbJ1FixYlJCSsXbtWX1EPdcjJAACGOoZh2tra2tra+ApAKpX6+vrydXZCSENDw+zZs994443333+fWy4SiaysrJKTk48ePcpXbD2zdu3aSZMmXb582czMrLM6RkZGGRkZY8eONTExGT169KFDh6ysrHbt2qVWq2kFV1fXEydObN68GYvW6AdyMgCAoc7MzKy0tPT06dN8B8KbrVu3VldXr1u3TqPcxMTkyJEjBgYG0dHRJSUlvMTWM59//nl8fHyHby0pd3f3Z8+eWVpasiVCodDZ2VmtVnPf1crl8rCwsBUrVrS2tvZvxICcDAAAhjiGYVJSUiZMmODo6Nh+q0KhWLNmTWNjY3h4uEbHsoFMLBZ3d5f6+vpbt255enrKZDJueUhISHl5+alTp/ouOugYcjIAgCHt5MmTbHd1mnNwS8rKyiIiIiwsLKysrIKCgkpLS+le27ZtoxWGDx+en5/v7+9vZmZmamo6bdq03NxcWmfTpk20Dvte8syZM7SErhzPHqepqSk3N5du0vJop58UFRU9ePBALpd3VmH9+vUBAQFXrlxZunSp9kM9evRo+fLlrq6uQqHQ0tJy5syZ586do5t0aVWqpqYmJiZm1KhRQqHQxsYmNDS0sLCw95epRUNDQ25ubnBwsL29/eHDhzW2jhs3jhDyzTff9GsMQAghDPQCISQ9PZ3vKACGtLCwsLCwML6jGBDoELme7TtnzhxCSHNzs0bJnDlzLl68qFKpzp49KxaLx48fz91LLpdLJBIfHx9aJz8/f+zYsUKh8Pz582wdiUQyefJk7l5eXl60d7yWOtS0adOGDRuWl5fXs4vS8d748ssvCSEff/yxRnl+fr5MJqP/rqmpcXZ2JoTQkYkMp48/q6qqysXFxc7OLjMzU6lUFhcXh4aGCgSCAwcOsHW6bNXKysqRI0fa2dmdOnWqsbHx2rVrU6ZMMTExuXjxYs8awcnJydDQUEuFjRs30nxg6tSpV65caV9BqVQSQvz8/Lo8F74TewnPyQAAoFNRUVE+Pj4SiWTGjBmBgYH5+fm1tbXcCk1NTXv27KF1vL29U1NTW1pali1b1idnb2tro99VfXK0zlRVVRFCNF7YabC2ts7IyDA2No6Ojr5582aHdRISEu7cubNz586goCBzc3M3N7e0tDQHB4eYmBiNwY9aWjUhIeHu3bvbt2+fNWuWVPr/sXf3cVFV+/7A1wbmiRkYEIQBhUQKuaGONHqCgp+IxWQiJImYYA9eupxOiVSYomRFGtfilJyXmijHlxliEF3wgFHHuHpfR4QO6gEzFbwgpgLKQwwD4giyf3+se/bZDTKMPG3Rz/svWXvN2muvmWa+7bXWdyt8fHy++uorlmUHvUU3ZMnJyQaD4dy5c97e3r6+vlyIxrG1tWUYho4SjCrEZAAAMKA5c+Zw/6Y3ihoaGvgV5HI5nduiZsyY4erqWlVVNSI/4UePHm1ra/P39x9+UybQGVuRSGS6mp+fX1paWldXV2RkZHd3d/8K+fn5hJCFCxdyJRKJZP78+d3d3UYTfyZGtaCgwMLCgp+XRKVS+fj4nDx58sqVK3d7aWYSi8Xe3t6ff/55WFjYxo0bf/jhB6MKVlZWd7xkGFmIyQAAYED8u0disZgQYpQyw87OzuglTk5OhJDr16+Pfu9GhlQqJYT09PQMWjM+Pj4qKurMmTNGKTMIIQaDQafTSaVSo9wTzs7OhJCmpiZ+4UCjShvp6+tTKpX8tLSnTp0ihFy4cGFoF2i+RYsWEUKKioqMynt7e4ewaQDu1lgvpQQAgPtJa2sry7IMw3AlNBqjkRkhxMLC4tatW/yXtLe3GzXCf/nYc3FxIYTQVVODyszMrKys3LNnD43kOBKJRKlU6nQ6vV7PD8vorKVKpTKncYlEYmdn19nZ2d3dPfZ7HWgHCCFtbW38wo6ODpZl6SjBqMJ9MgAAGLqbN2/SfPfUTz/91NDQoFaruZ9wFxeXq1evchWampp++eUXo0asra25uG3atGm7du0a5V7/xvTp0wkhZs4MKhSKb775Ri6X79ixw+jQ4sWLCSH8nBEGg6GkpEQmk2m1WjM7ExER0dvby+1dpbZs2eLu7j6yGcISExNjYmKMCouLi8lvp1YJIfTto6MEowoxGQAADJ1SqVy/fn1ZWVlXV9eJEydiYmLEYnF6ejpXISQkpKGhYdu2bZ2dnbW1tatXr+ZuoXEee+yxmpqay5cvl5WV1dXVBQYG0vLg4GAHB4fy8vJRvQS1Wu3k5FRVVWVmfR8fn4yMjP7lqampHh4eCQkJRUVFer2+pqZm+fLljY2N6enpdAbTHKmpqZ6enitXriwuLtbpdG1tbRkZGSkpKWlpadyds5iYGIZhLl68aGabA8nOzk5JSamvrzcYDPX19WvXrs3KytJoNLGxsfxqNBNHSEjIME8HgxNwz+d9gGDfL4DQkAuDM7RcGHRlOic6OrqsrIxfsmHDBva3Ox8XLlxIX6tWqydNmnT27FmtVmtjYyOTyebOnXvs2DF+++3t7bGxsS4uLjKZLCAgoKKiQqPR0HbWrl1L65w/fz4wMFAul7u5uW3fvp17bWBgoL29/ZDTQJj/2Vi/fr2VldXVq1fpn83Nzfzr1Wg0/V/y2muvGeXCYFm2paUlISHBw8NDJBIplUqtVltSUkIPmT+qNMnZ1KlTRSLRxIkTQ0JCDh8+zD9LcHCwQqHo7e01cUWFhYX9f/H5WTl0Ol1mZqZWq6WJ0BQKhUajSU1NvXHjhlFTkZGRkyZNunXr1qDDiN/EYWLYUd5jfH9jGCYnJ2fp0qVCdwTgwRUZGUkI+frrr4XuiPByc3OjoqLG8lt91qxZLS0to7cfcJjM/2zodDofH5/Q0NCdO3eOfr+Gpb293dXVNTo6evfu3WNwuqqqKl9f3+zs7GXLlg1aGb+Jw4S5S7gDfoZuofsydlpaWrhdTr6+vkYPUeEfZRhm9uzZQvXTtAfzvTPfpUuXXnnlFXd3d7FYzL2bmzZtGoNTsyxbWlr6+uuve3l5SSQSJyengIAAmoCUq/Prr7/u3LkzODh4woQJMpnskUceiY6ONn9ODYZMqVQWFhbm5eVt375d6L6YwrJsfHy8ra1t/yxio6Guri4iIiIpKcmcgAyGDzEZ3EFiYiLLskZPGuns7HzkkUf4WXNGyZidyIijoyPLsnS1cmVlZUJCQv+jXObuEydOjHH3zPRgvndmam5u9vPzO3XqVG5ubnt7O9tvOmlUVVdXBwQE1NTU5OXl6XS68vJyd3f3FStWrFmzhquzZs2aVatWhYeHnz17trW1dc+ePZWVlRqNpqCgYMz6+cDy9fU9ceJEcXFxR0eH0H0Z0LVr1+rq6kpKSszcyDlMGRkZmzdv3rx58xicCwhiMjAfy7J9fX1GqYmGSaFQcA/CG9UT3RWJROLg4JCRkXHgwAGh+jCyHpz3zrTMzMympqbPPvvMz8/P2tp69E50x8EhhFhZWeXm5s6cOVMqlU6dOnXv3r0ODg7btm0zGAxcnZUrV65evVqlUllbWwcGBmZnZ9++ffudd94Zvd4ODb0jW1VVdfXqVYZhkpOThe7RCJgyZUpRUZGtra3QHRmQSqU6duyYj4/P2Jxuy5YtuEM2lpCfDMxlY2Nj9KDc8X6igUil0v379z/77LNxcXEajcbLy0vAzoyIB+e9M+2nn34ihMyYMUOQs3t7extlJRWLxW5ubpWVlTdv3qR5oTIzM41epVarZTJZbW0t+9scYIJLTExMTEwUuhcA9xXcJwO4A61Wm5ycrNfrIyMjjRaWwfh148YNQohRmnUBtbe3X7hwwdfX18STFru6urq7u6dPn35PBWQAMBoQk42ugoICbh1xfX19VFSUnZ2dg4NDaGio0e0EuvnZ09NTLBbb29svWLDgyJEj/Ruprq5eunSpg4MD/TMzM5M7dOnSpaioKBsbGwcHhxUrVvz666/19fWLFi2ysbFxcXF59dVX9Xo9d7re3t6cnJynn35apVLJZLIZM2akp6ebmHLi94GLUezs7Jh+LCws6CYs06egEx9dXV2lpaX0hTT1zh1PZP74mB7ku/Lee++FhIScPn160Ef/4r271967gQbh4MGDhBCZTMYwzB3nFgft8JAHx0hHR0dpaWlYWJhKpdq3b5+JntM9gxs2bBjahQPAeDL26TfuJ8S8XCzh4eGEkPDw8OPHj3d2dh4+fFgmk82ZM4er0NjY6OHh4ezsXFhYqNPpqqurIyIiGIbh55KhjcydO/fIkSNdXV3l5eWWlpbNzc3coYiIiBMnTnR2dtKv+AULFoSHh//jH//Q6/V0d/ebb77JtUZT13z00UdtbW3Nzc1/+tOfLCws6PJwDs081P9Curu76Z9KpVKv13NHU1JSaJvmn0Iulz/55JMDjRh3IvPHx8Qgsyw7b968CRMmlJWVDfxesRUVFUqlkv67ubmZPh6Ybo5jeWv8OXjvjIzSe2eC+TmojPrG/nON/4cffsiVDNrh4QwOxe2YCwoKOn36tIkONzU1OTs7x8bGmnN17FDzk93HkLtujJn5mwgDwX+9w3JXMVlhYSFXsmTJEkII/VVmWfbll18mhBw4cICrcPPmTVdXV5lM1tTUxG/k22+/Haj9Q4cOcSV0Bej//M//cCUeHh7Tpk3j/iwsLAwKCuI3EhMTIxKJdDodV3JXv+s5OTkMw7z88st3dQozf9fNHx8Tg8yy7Ny5cwfNP8mPyViWLSsrE4lEcrn83Llz7J1iMrx3d7yiEX/vTBjZmGzQDg9ncDgGg+HcuXO///3vLS0tU1JS7linpaVl1qxZUVFRplOD8iEmM4KYbIwhJhsmrPEfO/wniNG7Lw0NDY6OjoQQmkd74cKFXAWJRDJ//vwvv/zy+++/f/HFF7ny3/3udwO1z8+Y5erq+vPPP/NLJk2axM9yFBoaapSwQK1WZ2Vl/fzzz/7+/mZeEfcg4R9//PGll176f//v//GfNzIip6DMHx8Tg0wIOXr06F2dlxDi5+eXlpa2evXqyMjIv//978PpG947amjv3ZgZtMMjMjhisdjb2/vzzz+/du3axo0b/f39n3rqKX6Frq4urVb76KOP7tu3z9LS8q4ugSZKBUIIfSgTBgTGC8RkY4e/jFcsFhNC6BoUg8Gg0+mkUqnR0mP6fLSmpiZ+oVwuH6h9/v5tCwsLS0tL/m5/S0tL/pIjnU73xz/+MT8//8qVK9zPM/nnIui78ssvv4SHh7u5uf3Xf/0Xva6RPcVdjc9Agzwc8fHxx48fz8nJeeONN1599dUh9w3vHWfM3ru7ZU6HR3D8CSGLFi3Kz88vKirix2S9vb30aTZffPHF3QZkADB+ISYTnkQiUSqVOp1Or9fzfwmuXbtGCBmlxICLFi3629/+lp6e/sILLzg6OjIMs3XrVrpo6a7a0ev1oaGhPT09RUVFEyZMuNtTMGZsJRNkfIxkZmZWVlbu2bNHKpUK3je8d6PHnA6P1OBwZySEtLW18Qvj4uIMBkN+fj63P+Dhhx/Oysry8/Mzp008ZoqD526NMfM/+XBH2Hd5T1i8eDEh5NChQ1yJwWAoKSmRyWRarXbET3f79u3S0lKVShUfHz9x4kT6X1F3d/cQ2lm2bNn58+e/+eYbLonXkiVLCgoKzDyFtbX1rVu36L+nTZu2a9euO55ojMenP4VC8c0338jl8h07dgjbN7x3o810h4czOImJiTExMUY1i4uLyW/nbd9///2ff/754MGDNFwDgAcHYrJ7QmpqqoeHR0JCQlFRkV6vr6mpWb58eWNjY3p6Op00GVmWlpZBQUFNTU2ffPJJS0tLd3f3kSNHhvDk3TfffPPbb7/dtWtXUFDQ0E7x2GOP1dTUXL58uaysrK6uLjAw8I4nGqnxCQ4OdnBwoEtM7paPjw9/xdWI981MD+x7N2ZMd3iYg5OdnZ2SklJfX28wGOrr69euXZuVlaXRaGJjY2mFvXv3fvDBBz/++KONjQ0/Tcm9nIkXAEaMoDsMxj0y2B4To6fpbdiwgf3tBNPChQtpzZaWloSEBA8PD5FIpFQqtVptSUnJHRvhv2v926ePa+Skpqb+7W9/45e89957LMs2NzfHxcW5ubmJRCJnZ+eXX3553bp1tIJGo/nkk0+MmqULnznR0dEmHviYn58/6Clo/8+fPx8YGCiXy93c3LZv386ybP8T3e34mBjkwMBAE/sum5ub+a/iOsn32muvGe27xHs3Nu+dCebsrbtj3zw9PfmFly9fHrTDQx4clmV1Ol1mZqZWq50yZYpYLFYoFBqNJjU19caNG1zj/L0FRkzncKGw79II9l2OMYJ9l8PDsHe5BgX4GIbJyclZunSp0B0BeHBhzRAnNzc3KioK3+ocfDbGGH4ThwlzlwAAAIQQcunSpbCwsI6OjpaWFm7i2NfX1+gBa/yjDMPwM9fca7799lsvL687PkyCZdnS0tLXX3/dy8tLIpE4OTkFBATQ/NhcnXXr1tGbrzA2EJMBAACQysrK2bNnh4SE2NraOjo6sixLlxNUVlYmJCTwa9KjXAZpE4sBBFRbWxsWFpaUlER3DfdXXV0dEBBQU1OTl5en0+nKy8vd3d1XrFixZs0ars6rr76alJT07rvvjlWvH3SIyQAAYCgUCoWJx4be++3zdXR0LFq06Pnnn3/jjTf45RKJxMHBISMj48CBA2PTk5Hy7rvvPvHEEydPnjRKtsdnZWWVm5s7c+ZMqVQ6derUvXv3Ojg4bNu2zWAw0Aqenp75+fmbN2/Ozc0dq44/0BCTAQDAg+7jjz9uamrauHGjUblUKt2/f7+FhUVcXFxNTY0gfRuaP//5z+vWrbvjrCXl7e3d09Njb2/PlYjFYjc3N4PBwJ+rVavVS5Ysefvtt3t7e0e3x4CYDAAAHnAsy2ZmZj7++OOurq79j2q12uTkZL1eHxkZabSw7F4mk8nu9iXt7e0XLlzw9fXlP1SDELJ48eIrV67wk/bBKEFMBgDwwGltbX3rrbc8PT3FYrG9vf2CBQuOHDlCD23atIkuXefmDb/77jtawj1+NC0tjWGYrq6u0tJSeojej6HlDMNMnjy5oqJi/vz5NjY21tbW8+bNKy0tHX77o6SqquratWtqtXqgCu+9915ISMjp06dXrVpluikTA1tQUMBtC6ivr4+KirKzs3NwcAgNDTXKP9fc3BwfH09zpkycODEiIqKysnL4l2lCR0dHaWlpWFiYSqXat2+f0dFZs2YRQr7//vtR7QMQgkw2w0OQiwVAaMhBxTEzP1ljY6OHh4ezs3NhYaFOp6uuro6IiGAYZvfu3VwduVz+5JNP8l+l0WiMkvP1r0Op1Wq5XO7v73/8+PHOzs6KioqZM2eKxeKjR4+OSPvz5s2bMGGCOQnbzPxsfPnll4SQjz76yKi8oqJCqVTSfzc3N7u5uRFC6M5ElrfGn2POwIaHhxNCwsPD6eAcPnxYJpPNmTOHq9DQ0PDQQw85OzsfOnRIr9efOXNm7ty5Uql0oMSKg5o0aZKlpaWJCh9++CGNB4KCgk6fPt2/gk6nI4QEBgYOei78Jg4T7pMBADxYkpKSLl68uHXr1tDQUFtbWy8vr+zsbBcXl/j4+IH26N2trq6uHTt2+Pv7y+Xy2bNnZ2Vl3bp1a/Xq1SPSeF9fH/0BG5HWCCGNjY2EEKMJOyOOjo65ubkikSguLu78+fN3rGP+wMbGxtLBeeqppxYuXFhRUdHS0sI1cunSpU8//fTZZ59VKBQ+Pj5fffUVy7KD3qIbsuTkZIPBcO7cOW9vb19fXy5E49g8R2QjAAAgAElEQVTa2jIMQ0cJRhViMgCABwt9qgH/mQESiWT+/Pnd3d0jNT8ll8vphBc1Y8YMV1fXqqqqEfldP3r0aFtbm7+///CbougqMZFIZLqan59fWlpaV1dXZGTkHR8ya/7A8p9wSm+/NTQ00D8LCgosLCxCQ0O5CiqVysfH5+TJk1euXLnbSzOTWCz29vb+/PPPw8LCNm7c+MMPPxhVsLKyGsJzdeFuISYDAHiAGAwGnU4nlUqNUiTQx482NTWNyFns7OyMSpycnAgh169fH5H2R5ZUKiWE9PT0DFozPj4+KirqzJkzRikzyF0OLP+enFgsJoT09fVxjfT19SmVSn5a2lOnThFCLly4MLQLNN+iRYsIIUVFRUblvb29Q9g0AHdrFFdNAgDAvUYikSiVSp1Op9fr+dEDnVxTqVT0TwsLi1u3bvFf2N7ebtQUwzADnaW1tZVlWX4FGo3RyGz47Y8sFxcXQghdNTWozMzMysrKPXv20EiOY+bAmiaRSOzs7Do7O7u7u0d1W4OJDhBC2tra+IUdHR0sy9JRglGF+2QAAA+WxYsXE0L4qQ0MBkNJSYlMJtNqtbTExcXl6tWrXIWmpqZffvnFqB1ra2surpo2bdquXbu4Qzdv3qRJ8KmffvqpoaFBrVZzv+vDbH9kTZ8+nRBi5sygQqH45ptv5HL5jh07jA6ZM7CDioiI6O3t5bapUlu2bHF3dx/ZDGGJiYkxMTFGhcXFxeS3U6uEEPpO0VGCUYWYDADgwZKamurh4ZGQkFBUVKTX62tqapYvX97Y2Jienk4n2gghISEhDQ0N27Zt6+zsrK2tXb16NXeLi/PYY4/V1NRcvny5rKysrq4uMDCQO6RUKtevX19WVtbV1XXixImYmBixWJyens5VGE77wcHBDg4O5eXlIzUgarXaycmpqqrKzPo+Pj4ZGRn9y80Z2EGlpqZ6enquXLmyuLhYp9O1tbVlZGSkpKSkpaVxd85iYmIYhrl48aKZbQ4kOzs7JSWlvr7eYDDU19evXbs2KytLo9HExsbyq9FMHCEhIcM8HQxOwD2f9wGCfb8AQkMuDI6ZuTBYlm1paUlISPDw8BCJREqlUqvVlpSU8Cu0t7fHxsa6uLjIZLKAgICKigqNRkN/NdauXUvrnD9/PjAwUC6Xu7m5bd++nXutWq2eNGnS2bNntVqtjY2NTCabO3fusWPHRqr9wMBAe3t7c3JDmP/ZWL9+vZWV1dWrV+mfzc3N/B9KjUbT/yWvvfaaUS4M1uTAlpWV8dvcsGED+9utowsXLqQ1aZKzqVOnikSiiRMnhoSEHD58mH+W4OBghULR29tr4ooKCwv7/+Lzs3LodLrMzEytVksToSkUCo1Gk5qaeuPGDaOmIiMjJ02adOvWrUGHEb+Jw8SwI7ed+AHEMExOTs7SpUuF7gjAgysyMpIQ8vXXXwvdEeHl5uZGRUUJ/q0+a9aslpaW0dskaD7zPxs6nc7Hxyc0NHTnzp2j369haW9vd3V1jY6O3r179xicrqqqytfXNzs7e9myZYNWxm/iMGHuEgAAHnRKpbKwsDAvL2/79u1C98UUlmXj4+NtbW37ZxEbDXV1dREREUlJSeYEZDB8iMkAAACIr6/viRMniouLOzo6hO7LgK5du1ZXV1dSUmLmRs5hysjI2Lx58+bNm8fgXEAQkwEAwEihz6msqqq6evUqwzDJyclC9+juTJkypaioyNbWVuiODEilUh07dszHx2dsTrdlyxbcIRtLyE8GAAAjIzExMTExUeheAIxXuE8GAAAAIDzEZAAAAADCQ0wGAAAAIDzEZAAAAADCQ0wGAAAAIDzk8R8WhmGE7gIAAMC9Ann8hwO5MIaFPl0OAIQVFRWVkJDg7+8vdEcAHnRPPPGE0F0Yx3CfDADGPTxlDwDuA1hPBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8xGQAAAAAwrMSugMAAHftwIEDer2eX/LDDz+0t7dzfy5evHjixIlj3i8AgKFjWJYVug8AAHfn5Zdf/uKLL0QiEf2Tfo8xDEMIuX37tkKhuH79ukQiEbKLAAB3CXOXADD+vPDCC4SQnn/q7e3t7e2l/7a0tIyMjERABgDjDu6TAcD409vb6+zs3NbWdsejJSUlwcHBY9wlAIBhwn0yABh/rKysXnjhBW7uks/R0XHu3Llj3yUAgGFCTAYA49ILL7zQ09NjVCgSiVasWGFpaSlIlwAAhgNzlwAwLrEs6+7ufuXKFaPyv//973PmzBGkSwAAw4H7ZAAwLjEMExMTYzR96ebmNnv2bKG6BAAwHIjJAGC8Mpq+FIlEL7/8Ms2IAQAw7mDuEgDGMW9v7+rqau7PM2fO+Pj4CNgfAIAhw30yABjHVqxYwU1fPvroowjIAGD8QkwGAONYTExMb28vIUQkEr300ktCdwcAYOgwdwkA49vs2bNPnjzJMEx9fb27u7vQ3QEAGCLcJwOA8e3FF18khDz++OMIyABgXLMSugNwr4uMjBS6CwCm3Lx5k2EYg8GAzyrc49566y1/f3+hewH3Ltwng0Hk5eX1T8sJ40V5eXl5ebnQvRhdUqnU2dl58uTJpqtduXIlLy9vbLo0LjwIn417Sl5e3uXLl4XuBdzTcJ8MBvfmm28uXbpU6F7AUNBbR19//bXQHRld//u///vwww+brpObmxsVFXXfD4X5HpDPxr0DmfNgULhPBgDj3qABGQDAvQ8xGQAAAIDwEJMBAAAACA8xGQAAAIDwEJMBAMDgLl26FBYW1tHR0dLSwvyTr6/vzZs3+dX4RxmGmT17tlAdHtS3337r5eVlZXWHvW4sy5aWlr7++uteXl4SicTJySkgICArK4ufZX3dunU5OTlj2F+4/yEmAwAwpbOz85FHHgkNDRW6I0KqrKycPXt2SEiIra2to6Mjy7IVFRW0PCEhgV+THi0rK3NwcGBZ9sSJEwJ12ZTa2tqwsLCkpKRr167dsUJ1dXVAQEBNTU1eXp5OpysvL3d3d1+xYsWaNWu4Oq+++mpSUtK77747Vr2G+x9iMgAAU1iW7evr6+vrE6oDCoUiICBAqLMTQjo6OhYtWvT888+/8cYb/HKJROLg4JCRkXHgwAGh+jY077777hNPPHHy5EkbG5uB6lhZWeXm5s6cOVMqlU6dOnXv3r0ODg7btm0zGAy0gqenZ35+/ubNm3Nzc8eq43CfQ0wGAGCKjY1NbW3tt99+K3RHBPPxxx83NTVt3LjRqFwqle7fv9/CwiIuLq6mpkaQvg3Nn//853Xr1t1x1pLy9vbu6emxt7fnSsRisZubm8Fg4M/VqtXqJUuWvP322729vaPbY3gwICYDAIABsSybmZn5+OOPu7q69j+q1WqTk5P1en1kZKTRwrJ7mUwmu9uXtLe3X7hwwdfXV6lU8ssXL1585cqVQ4cOjVzv4MGFmAwAYEAFBQXccnUac/BL6uvro6Ki7OzsHBwcQkNDa2tr6avS0tJohcmTJ1dUVMyfP9/Gxsba2nrevHmlpaW0zqZNm2gdbl7yu+++oyWOjo78drq6ukpLS+khE7d2RklVVdW1a9fUavVAFd57772QkJDTp0+vWrXKdFOtra1vvfWWp6enWCy2t7dfsGDBkSNH6CFzRpVqbm6Oj4+fMmWKWCyeOHFiREREZWXl8C/ThI6OjtLS0rCwMJVKtW/fPqOjs2bNIoR8//33o9oHeFCwACYRQnJycoTuBQzRkiVLlixZInQv7gl0i9zQXhseHk4I6e7uNioJDw8/fvx4Z2fn4cOHZTLZnDlz+K9Sq9Vyudzf35/WqaiomDlzplgsPnr0KFdHLpc/+eST/FdpNBq6Ot5EHWrevHkTJkwoKysb2kWZ+dn48ssvCSEfffSRUXlFRYVSqaT/bm5udnNzI4TQnYksb40/p7Gx0cPDw9nZubCwUKfTVVdXR0REMAyze/durs6go9rQ0PDQQw85OzsfOnRIr9efOXNm7ty5Uqn0+PHjQxuESZMmWVpamqjw4Ycf0t/KoKCg06dP96+g0+kIIYGBgYOeC9+lMCjcJwMAGKLY2Fh/f3+5XP7UU08tXLiwoqKipaWFX6Grq2vHjh20zuzZs7Oysm7durV69eoROXtfXx/9Hh+R1gbS2NhICDGasDPi6OiYm5srEoni4uLOnz9/xzpJSUkXL17cunVraGiora2tl5dXdna2i4tLfHy80eZHE6OalJR06dKlTz/99Nlnn1UoFD4+Pl999RXLsoPeohuy5ORkg8Fw7tw5b29vX19fLkTj2NraMgxDRwlgmBCTAQAM0Zw5c7h/0xtFDQ0N/ApyuZzObVEzZsxwdXWtqqoakZ/wo0ePtrW1+fv7D78pE+iMrUgkMl3Nz88vLS2tq6srMjKyu7u7f4X8/HxCyMKFC7kSiUQyf/787u5uo4k/E6NaUFBgYWHBz0uiUql8fHxOnjx55cqVu700M4nFYm9v788//zwsLGzjxo0//PCDUQUrK6s7XjLA3UJMBgAwRPy7R2KxmBBilDLDzs7O6CVOTk6EkOvXr49+70aGVColhPT09AxaMz4+Pioq6syZM0YpMwghBoNBp9NJpVKj3BPOzs6EkKamJn7hQKNKG+nr61Mqlfy0tKdOnSKEXLhwYWgXaL5FixYRQoqKiozKe3t7h7BpAKC/sV4uCgDw4GhtbWVZlmEYroRGYzQyI4RYWFjcunWL/5L29najRvgvH3suLi6EELpqalCZmZmVlZV79uyhkRxHIpEolUqdTqfX6/lhGZ21VKlU5jQukUjs7Ow6Ozu7u7vHfq8D7QAhpK2tjV/Y0dHBsiwdJYBhwn0yAIDRcvPmTZrvnvrpp58aGhrUajX3E+7i4nL16lWuQlNT0y+//GLUiLW1NRe3TZs2bdeuXaPc69+YPn06IcTMmUGFQvHNN9/I5fIdO3YYHVq8eDEhhJ8zwmAwlJSUyGQyrVZrZmciIiJ6e3u5vavUli1b3N3dRzZDWGJiYkxMjFFhcXEx+e3UKiGEvn10lACGCTEZAMBoUSqV69evLysr6+rqOnHiRExMjFgsTk9P5yqEhIQ0NDRs27ats7OztrZ29erV3C00zmOPPVZTU3P58uWysrK6urrAwEBaHhwc7ODgUF5ePqqXoFarnZycqqqqzKzv4+OTkZHRvzw1NdXDwyMhIaGoqEiv19fU1CxfvryxsTE9PZ3OYJojNTXV09Nz5cqVxcXFOp2ura0tIyMjJSUlLS2Nu3MWExPDMMzFixfNbHMg2dnZKSkp9fX1BoOhvr5+7dq1WVlZGo0mNjaWX41m4ggJCRnm6QAIQS4MGAzB/u3xDLkwOEPLhUFXpnOio6PLysr4JRs2bGB/u/Nx4cKF9LVqtXrSpElnz57VarU2NjYymWzu3LnHjh3jt9/e3h4bG+vi4iKTyQICAioqKjQaDW1n7dq1tM758+cDAwPlcrmbm9v27du51wYGBtrb2w85DYT5n43169dbWVldvXqV/tnc3My/Xo1G0/8lr732mlEuDJZlW1paEhISPDw8RCKRUqnUarUlJSX0kPmjSpOcTZ06VSQSTZw4MSQk5PDhw/yzBAcHKxSK3t5eE1dUWFjY/9eQn5VDp9NlZmZqtVqaCE2hUGg0mtTU1Bs3bhg1FRkZOWnSpFu3bg06jPguhUEx7Cjvo4bxjmGYnJycpUuXCt0RGIrIyEhCyNdffy10R4SXm5sbFRU1lt94s2bNamlpGb39gMNk/mdDp9P5+PiEhobu3Llz9Ps1LO3t7a6urtHR0bt37x6D01VVVfn6+mZnZy9btmzQyvguhUFh7hIAAExRKpWFhYV5eXnbt28Xui+msCwbHx9va2vbP4vYaKirq4uIiEhKSjInIAMwB2IyGHlfffUV3aNutPdqXFAoFPxt9hYWFvb29mq1+g9/+MPJkyeF7t3ounTp0iuvvOLu7i4Wi7kR2LRp0xicmmXZ0tLS119/3cvLSyKRODk5BQQE0KTwXJ1ff/11586dwcHBEyZMkMlkjzzySHR0tPnrnGA4fH19T5w4UVxc3NHRIXRfBnTt2rW6urqSkhIzN3IOU0ZGxubNmzdv3jwG54IHBGIyGHnLli1jWXb+/PlCd2QoOjs7//GPfxBCwsPDWZbt6ek5f/58SkrK+fPnZ8+e/corr9y4cUPoPo6K5uZmPz+/U6dO5ebmtre3s/2W+Iyq6urqgICAmpqavLw8nU5XXl7u7u6+YsWKNWvWcHXWrFmzatWq8PDws2fPtra27tmzp7KyUqPRFBQUjFk/zUSfU1lVVXX16lWGYZKTk4Xu0QiYMmVKUVGRra2t0B0ZkEqlOnbsmI+Pz9icbsuWLbhDBiMLMRmAKZaWls7OzuHh4f/93//9zjvv7N2794UXXrgvV2FmZmY2NTV99tlnfn5+1tbWo3cihULBPXWbz8rKKjc3d+bMmVKpdOrUqXv37nVwcNi2bZvBYODqrFy5cvXq1SqVytraOjAwMDs7+/bt2++8887o9XZoEhMT+et2x+ZeIwCMd4jJAMz1n//5n48//vhf/vKXr776Sui+jLyffvqJEDJjxgxBzu7t7d3T02Nvb8+ViMViNzc3g8FAn+1DCMnMzDRKsqBWq2UyWW1t7X0ZJQPAgwYxGYC5GIahD43pnw/zPkDnZI0efSOg9vb2Cxcu+Pr6mnj6dVdXV3d39/Tp04XNdA8AMCIQk8HIOH/+/HPPPadUKuVyeWBg4LFjx/rXaW5ujo+Pp/l+Jk6cGBERQdMtEkIKCgq4ReX19fVRUVF2dnYODg6hoaG1tbVcCwaDYePGjd7e3tbW1hMmTFi0aNFf/vKX27dvm3OKEUEn3crLy7nH/90HF0X7efDgQUKITCZjGOaOc4sUzQ7l6ekpFovt7e0XLFhw5MgR7mhvb29OTs7TTz+tUqlkMtmMGTPS09O5R0DSVVZdXV2lpaV0WO74hJyOjo7S0tKwsDCVSrVv3z4TPad5HDZs2DC0CwcAuLeMbTo0GH+IGXkOL1y4YGdnN2nSpL/+9a96vf706dMhISFTpkyRSCRcnYaGhoceesjZ2fnQoUN6vf7MmTNz586VSqX8jJfh4eGEkPDw8OPHj3d2dh4+fFgmk82ZM4erEBsbq1Qq//rXv964caOpqSkxMZEQcuTIEfNPMW/evAkTJpSVlZm4HP4afyPd3d30P5yGhoZ756JMMD8vKO1nd3c3V0LX+H/44YdcSWNjo4eHh7Ozc2FhoU6nq66ujoiIYBiGS7ZJU3F+9NFHbW1tzc3Nf/rTnywsLIwWV8nl8ieffHKgbnBZDIKCgk6fPm2iw01NTc7OzrGxseZcHTvUnLH3MeQTHmPmfJfCAw7fUDAIc75HaPLJvLw8ruTq1asSiYQfk7300kuEkP3793MljY2NEomEnwSchgWFhYVcyZIlSwghzc3N9E8PD48nnniCf2ovLy8ufDHnFHPnzh009bmJmIzbdEljsnvkokwY2Zjs5ZdfJoQcOHCAK7l586arq6tMJmtqamJZtrCwMCgoiN9sTEyMSCTS6XRciemYjGVZg8Fw7ty53//+95aWlikpKXes09LSMmvWrKioKNPp2vkQkxlBTDbGEJPBoPANBYMw53uELkLS6/X8whkzZvBjMqVSaWFhwf9tZln2scceI4RcvnyZ/knDAvrrTr355puEkKqqKvrna6+9Rgh59dVXy8rK+v8Ym3MKc5iIyeico0gkoo9SufcvamRjMrq0q6Ojg//CFStWEEK++OKLOzb7ySefEEL4QfCgMRmHPrXa6Mk5LMt2dnZqNJrly5ebH5Cx/4zJAASEmAxMu8NiDoC7YjAY9Hq9VCpVKBT8cicnp5qaGq6OTqcjhNxxvfaFCxcmT57M/cmvIxaLCSHcgqTt27f7+/t/8cUXNPlZYGBgXFwc/eW+q1MMGV0n5+/vLxKJ7puLMhPtjFQqNdoHQB8g3dTURAjR6XR//OMf8/Pzr1y50t7eztUZWlK3RYsW5efnFxUVPfXUU1xhb28vfcLgF198YWlpebdtIjLjfPbZZ4QQ+n8IMAaioqKE7gLc6xCTwXBJJBIbGxu9Xt/Z2ckPy9ra2vh17OzsOjs7u7u777is20wMw6xYsWLFihU9PT1Hjx5NS0uLiIj44x//+NZbb43UKUzo6+ujz5Z5/fXXyf1yUeaTSCRKpVKn0+n1en5Ydu3aNUIIzZy+aNGiv/3tb+np6S+88IKjoyPDMFu3bn3zzTdZXq4K8/dISiQS8tsPEiEkLi7OYDDk5+dzA/Lwww9nZWX5+fmZ0yaeNsihOyQwIGMGMRkMCvsuYQQsWLCAEPLdd99xJS0tLdXV1fw6ERERvb29paWl/MItW7a4u7v39vaaeSI7O7vz588TQkQi0dNPP003DB46dGgET2FCUlLS3//+98WLF9P1cyN1RmEv6q7Qu3dc3wghBoOhpKREJpNptdrbt2+XlpaqVKr4+PiJEyfS2IvbFcGxtra+desW/fe0adN27dpFCElMTIyJiTGqWVxcTAiZM2cOV/L+++///PPPBw8epOEaAMD9BDEZjICPPvpowoQJCQkJhw8f7uzsPHv2bExMjNFUZmpqqqen58qVK4uLi3U6XVtbW0ZGRkpKSlpa2l3dAfr9739/+vRpg8Fw/fr1jz/+mGXZ4OBg808RHBzs4OBQXl5u5un6+vquX79+8ODB+fPnf/zxxytXrty/fz93p+ceuagxk5qa6uHhkZCQUFRUpNfra2pqli9f3tjYmJ6e7uzsbGlpGRQU1NTU9Mknn7S0tHR3dx85cmTnzp1GjTz22GM1NTWXL18uKyurq6sLDAyk5dnZ2SkpKfX19QaDob6+fu3atVlZWRqNJjY2llbYu3fvBx988OOPP9rY2PCfScpPLAIAMI4JvJ4N7nnEvHWp1dXVzz33nK2tLU30UFRUxD3v8t///d9pHZraaurUqSKRaOLEiSEhIdzybaNHK27YsIH9bWb2hQsXsixbWVkZFxf3b//2bzSVl5+f3+7du/v6+rhumDgFFRgYaHrfpVwu55+XYRilUjljxozXXnvt5MmT/evfCxdlgjlr/PPz8/m9io6OZlnW09OTX8jtJ2hpaUlISPDw8BCJREqlUqvVlpSUcE01NzfHxcW5ubmJRCJnZ+eXX3553bp1tAVuo+j58+cDAwPlcrmbm9v27dtpoU6ny8zM1Gq1NA2bQqHQaDSpqak3btzgGl+4cOFA32Omk5tQ2HdpBPsux5iZ36XwIGNYPJMETGIYJicnB4tOxik6zUpXDj3gcnNzo6Ki8I3HwWdjjOG7FAaFuUsAABjcpUuXwsLCOjo6WlpauIljX19f7oGkFP8owzCzZ88WqsMDYVm2tLT09ddf9/LykkgkTk5OAQEBWVlZ/Hj9119/3blzZ3Bw8IQJE2Qy2SOPPBIdHV1VVcVvZ926ddjGCyMLMRkAAAyisrJy9uzZISEhtra2jo6OLMtWVFTQ8oSEBH5NerSsrMzBwYFl2RMnTgjU5QFVV1cHBATU1NTk5eXpdLry8nJ3d/cVK1asWbOGq7NmzZpVq1aFh4efPXu2tbV1z549lZWVGo2moKCAq/Pqq68mJSW9++67QlwE3J8QkwEAjDyFQmHisaH3fvt8HR0dixYtev7559944w1+uUQicXBwyMjIOHDgwNj0ZKRYWVnl5ubOnDlTKpVOnTp17969Dg4O27ZtMxgMXJ2VK1euXr1apVJZW1sHBgZmZ2ffvn37nXfe4Sp4enrm5+dv3rw5NzdXiIuA+xBiMgAAMOXjjz9uamrauHGjUblUKt2/f7+FhUVcXByXIPre5+3t3dPTY29vz5WIxWI3NzeDwcDNw2ZmZmZkZPBfpVarZTJZbW0tf4pTrVYvWbLk7bffHvvENHBfQkwGAAADYlk2MzPz8ccfd3V17X9Uq9UmJyfr9frIyEijhWXjSHt7+4ULF3x9fe/4wAyqq6uru7t7+vTpRkmPFy9efOXKFX7SPoAhQ0wGAPAbNPmIp6enWCy2t7dfsGDBkSNH6KFNmzbRpevcvOF3331HSxwdHWlJWloawzBdXV2lpaX0EM0kR8sZhpk8eXJFRcX8+fNtbGysra3nzZvH5QQeTvujpKqq6tq1a2q1eqAK7733XkhIyOnTp1etWmW6KRMDS1MlU/X19VFRUXZ2dg4ODqGhoUb555qbm+Pj42nOlIkTJ0ZERFRWVg756jo6OkpLS8PCwlQq1b59+0zUpBtUN2zYYFQ+a9YsQsj3338/5D4A/ItgWThgnCDIqTOeIQcVx8z8ZI2NjR4eHs7OzoWFhTqdrrq6OiIigmGY3bt3c3X6P0Zdo9HQJe0m6lBqtVoul/v7+x8/fryzs7OiomLmzJlisfjo0aMj0v68efMmTJhgTsI2Mz8bX375JSHko48+MiqvqKhQKpX0383NzW5uboQQunuR5a3x55gzsOHh4YSQ8PBwOjiHDx+m+Q65Cg0NDQ899JCzs/OhQ4f0ev2ZM2fmzp0rlUpNZBw04cMPP6S/g0FBQadPnzZRs6mpydnZOTY2tv8h+kTawMDAQU+H71IYFO6TAQD8S1JS0sWLF7du3RoaGmpra+vl5ZWdne3i4hIfH0+f7Dl8XV1dO3bs8Pf3l8vls2fPzsrKunXr1urVq0ekcS7b8Ii0RghpbGwkhJiY1Kdg+JIAACAASURBVCOEODo65ubmikSiuLg4+qCw/swf2NjYWDo4Tz311MKFCysqKlpaWrhGLl269Omnnz777LMKhcLHx+err75iWXbQW3R3lJycbDAYzp075+3t7evry4VoRlpbW5955pmgoKD+D6UghNja2jIMQ0cJYJgQkwEA/At9qgH/mQESiWT+/Pnd3d0jNT8ll8vphBc1Y8YMV1fXqqqqEfldP3r0aFtbm7+///CbougqMZFIZLqan59fWlpaV1dXZGRk/4eckrsZWP4TTuntt4aGBvpnQUGBhYVFaGgoV0GlUvn4+Jw8efLKlSt3e2mEELFY7O3t/fnnn4eFhW3cuPGHH34wqtDV1aXVah999NH9+/dbWlresRErK6s7XjLA3UJMBgDwfwwGg06nk0qlNjY2/HJnZ2dCSFNT04icxc7OzqjEycmJEHL9+vURaX9kSaVSQkhPT8+gNePj46Oios6cOWOUMoPc5cDy78mJxWJCSF9fH9dIX1+fUqnkp6U9deoUIeTChQtDu0Bq0aJFhJCioiJ+YW9vb2Rk5KRJk7744ouBAjJaTSaTDefsANRYP8MYAOCeJZFIlEqlTqfT6/X86IFOrqlUKvqnhYXFrVu3+C9sb283aspodx5fa2sry7L8CjQao5HZ8NsfWS4uLoQQumpqUJmZmZWVlXv27KGRHMfMgTVNIpHY2dl1dnZ2d3eP+LYGiURCCGlra+MXxsXFGQyG/Px87nQPP/xwVlaWn58fV6ejo4NlWTpKAMOE+2QAAP+yePFiQgg/tYHBYCgpKZHJZFqtlpa4uLhcvXqVq9DU1PTLL78YtWNtbc3FVdOmTdu1axd36ObNmzQJPvXTTz81NDSo1Wrud32Y7Y+s6dOnE0LMnBlUKBTffPONXC7fsWOH0SFzBnZQERERvb293DZVasuWLe7u7uZnCEtMTIyJiTEqLC4uJr+dNn3//fd//vnngwcP0nBtIPSdoqMEMEyIyQAA/iU1NdXDwyMhIaGoqEiv19fU1CxfvryxsTE9PZ1OtBFCQkJCGhoatm3b1tnZWVtbu3r1au4WF+exxx6rqam5fPlyWVlZXV1dYGAgd0ipVK5fv76srKyrq+vEiRMxMTFisTg9PZ2rMJz2g4ODHRwcysvLR2pA1Gq1k5OT0aMeTfDx8THKtkqZM7CDSk1N9fT0XLlyZXFxsU6na2try8jISElJSUtL425lxcTEMAxz8eJFE+1kZ2enpKTU19cbDIb6+vq1a9dmZWVpNJrY2FhaYe/evR988MGPP/5oY2PDnyc1SsxBCKGZOEJCQsy8BABTBNzzCeMCwf7t8Qy5MDhm5sJgWbalpSUhIcHDw0MkEimVSq1WW1JSwq/Q3t4eGxvr4uIik8kCAgIqKio0Gg39Rl27di2tc/78+cDAQLlc7ubmtn37du61arV60qRJZ8+e1Wq1NjY2Mpls7ty5x44dG6n2AwMD7e3tzckNYf5nY/369VZWVlevXqV/Njc3839ENBpN/5e89tprRrkwWJMDW1ZWxm9zw4YN7G+3ji5cuJDWpEnOpk6dKhKJJk6cGBIScvjwYf5ZgoODFQpFb2/vQJej0+kyMzO1Wi1NcqZQKDQaTWpq6o0bN7g6/L0IRozyjNAFZ7du3Rp0GPFdCoNi2JHbMg33JYZhcnJyli5dKnRHYCgiIyPJP9NdPuByc3OjoqIE/8abNWtWS0vL0DYJjizzPxs6nc7Hxyc0NPSOySDuKe3t7a6urtHR0bt37x6D01VVVfn6+mZnZy9btmzQyvguhUFh7hIAAExRKpWFhYV5eXnbt28Xui+msCwbHx9va2s7UKaxkVVXVxcREZGUlGROQAZgDsRkAAAwCF9f3xMnThQXF3d0dAjdlwFdu3atrq6upKTEzI2cw5SRkbF58+bNmzePwbngAYGYDABgLNDnVFZVVV29epVhmOTkZKF7dHemTJlSVFRka2srdEcGpFKpjh075uPjMzan27JlC+6QwchCfjIAgLGQmJiYmJgodC8A4N6F+2QAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8rPGHwRml2IZxhOYmzc3NFbojwqMfYwwFB58NgHsN8vjDIBiGEboLAAD3A+TxB9MQkwHAuIen1gDAfQDryQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEh5gMAAAAQHiIyQAAAACEx7AsK3QfAADuTlxcXHV1NffnqVOnPDw87O3t6Z+WlpZffPHF5MmTBeodAMBQWAndAQCAu+bs7Lxr1y5+yenTp7l/T506FQEZAIw7mLsEgPFn+fLlAx0Si8Uvv/zyGPYFAGBkYO4SAMal6dOnnz179o7fYNXV1V5eXmPfJQCA4cB9MgAYl1588UVLS0ujQoZh1Go1AjIAGI8QkwHAuPTCCy/cvn3bqNDS0vKll14SpD8AAMOEuUsAGK+eeOKJH3/8sa+vjythGOby5cuTJk0SsFcAAEOD+2QAMF6tWLGCYRjuTwsLi4CAAARkADBOISYDgPEqMjKS/yfDMC+++KJQnQEAGCbEZAAwXjk6Os6fP59b6c8wzOLFi4XtEgDAkCEmA4BxLCYmhi6KtbS01Gq1Dg4OQvcIAGCIEJMBwDgWEREhFosJISzLxsTECN0dAIChQ0wGAOOYXC4PDQ0lhIjF4kWLFgndHQCAoUNMBgDjW3R0NCFk8eLFcrlc6L4AAAwd8pPBUERGRubl5QndCwCAexR+W2EIrITuAIxXfn5+b775ptC9gH+JiopKSEjw9/cXuiMCyMrKWrZsmZXV/32hffbZZ4QQfD45D/JnY+yVlZVt3bpV6F7AuIT7ZDAUNC/U119/LXRH4F8YhsnJyVm6dKnQHRHAzZs3pVIp9yc+n0Ye5M/G2MvNzY2KisJvKwwB1pMBwLjHD8gAAMYpxGQAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAA/3Lp0qWwsLCOjo6Wlhbmn3x9fW/evMmvxj/KMMzs2bOF6vBAWJYtLS19/fXXvby8JBKJk5NTQEBAVlYWf/X9r7/+unPnzuDg4AkTJshkskceeSQ6Orqqqorfzrp163Jycsa8+/AgQkwGAPB/Ojs7H3nkEfpggAdTZWXl7NmzQ0JCbG1tHR0dWZatqKig5QkJCfya9GhZWZmDgwPLsidOnBCoywOqrq4OCAioqanJy8vT6XTl5eXu7u4rVqxYs2YNV2fNmjWrVq0KDw8/e/Zsa2vrnj17KisrNRpNQUEBV+fVV19NSkp69913hbgIeLAgJgMA+D8sy/b19fX19QnVAYVCERAQINTZOzo6Fi1a9Pzzz7/xxhv8colE4uDgkJGRceDAAaH6NjRWVla5ubkzZ86USqVTp07du3evg4PDtm3bDAYDV2flypWrV69WqVTW1taBgYHZ2dm3b99+5513uAqenp75+fmbN2/Ozc0V4iLgAYKYDADg/9jY2NTW1n777bdCd0QYH3/8cVNT08aNG43KpVLp/v37LSws4uLiampqBOnbEHh7e/f09Njb23MlYrHYzc3NYDBw87CZmZkZGRn8V6nVaplMVltby5/iVKvVS5Ysefvtt3t7e8em8/BgQkwGAACEZdnMzMzHH3/c1dW1/1GtVpucnKzX6yMjI40Wlo0j7e3tFy5c8PX1VSqVA9Xp6urq7u6ePn06wzD88sWLF1+5cuXQoUOj3014cCEmAwAghJCCggJuxToNO/gl9fX1UVFRdnZ2Dg4OoaGhtbW19FVpaWm0wuTJkysqKubPn29jY2NtbT1v3rzS0lJaZ9OmTbQONy/53Xff0RJHR0d+O11dXaWlpfQQ96iosVFVVXXt2jW1Wj1Qhffeey8kJOT06dOrVq0y3VRra+tbb73l6ekpFovt7e0XLFhw5MgResicIaWam5vj4+OnTJkiFosnTpwYERFRWVk55Kvr6OgoLS0NCwtTqVT79u0zUZM+/mHDhg1G5bNmzSKEfP/990PuA8DgWIC7t2TJkiVLlgjdC/gNQkhOTo7QvbgnDOfzGR4eTgjp7u42KgkPDz9+/HhnZ+fhw4dlMtmcOXP4r1Kr1XK53N/fn9apqKiYOXOmWCw+evQoV0culz/55JP8V2k0GrpA3kQdat68eRMmTCgrKxvaRZnz2fjyyy8JIR999JFReUVFhVKppP9ubm52c3MjhNDdiyxvjT+nsbHRw8PD2dm5sLBQp9NVV1dHREQwDLN7926uzqBD2tDQ8NBDDzk7Ox86dEiv1585c2bu3LlSqfT48eNDuPwPP/yQ/t4FBQWdPn3aRM2mpiZnZ+fY2Nj+h3Q6HSEkMDBw0NPRTZpD6CcA7pMBAAwuNjbW399fLpc/9dRTCxcurKioaGlp4Vfo6urasWMHrTN79uysrKxbt26tXr16RM7e19dHv7JHpLU7amxsJISYmNQjhDg6Oubm5opEori4uPPnz9+xTlJS0sWLF7du3RoaGmpra+vl5ZWdne3i4hIfH3/t2jV+TRNDmpSUdOnSpU8//fTZZ59VKBQ+Pj5fffUVy7KD3qK7o+TkZIPBcO7cOW9vb19fXy5EM9La2vrMM88EBQXt3Lmz/1FbW1uGYegoAYwSxGQAAIObM2cO9296r6ihoYFfQS6X0+ktasaMGa6urlVVVSPyK3706NG2tjZ/f//hNzUQOl0rEolMV/Pz80tLS+vq6oqMjOzu7u5fIT8/nxCycOFCrkQikcyfP7+7u9to4s/EkBYUFFhYWPCTkqhUKh8fn5MnT165cuVuL40QIhaLvb29P//887CwsI0bN/7www9GFbq6urRa7aOPPrp//35LS8s7NmJlZXXHSwYYKYjJAAAGx7+BJBaLCSFGKTPs7OyMXuLk5EQIuX79+uj3bgTQ57j39PQMWjM+Pj4qKurMmTNGKTMIIQaDQafTSaVSGxsbfrmzszMhpKmpiV840JDSRvr6+pRKJT8t7alTpwghFy5cGNoFUosWLSKEFBUV8Qt7e3sjIyMnTZr0xRdfDBSQ0WoymWw4ZwcwbUzXkAIA3K9aW1tZluVv1qPRGI3MCCEWFha3bt3iv6S9vd2oEaO9fmPJxcWFEEJXTQ0qMzOzsrJyz549NJLjSCQSpVKp0+n0ej0/LKOzliqVypzGJRKJnZ1dZ2dnd3f3iG90kEgkhJC2tjZ+YVxcnMFgyM/P50738MMPZ2Vl+fn5cXU6OjpYlqWjBDBKcJ8MAGAE3Lx5k6a8p3766aeGhga1Ws39iru4uFy9epWr0NTU9Msvvxg1Ym1tzcVt06ZN27Vr1yj3+l+mT59OCDFzZlChUHzzzTdyuXzHjh1GhxYvXkwI4eeMMBgMJSUlMplMq9Wa2ZmIiIje3l5u4yq1ZcsWd3d38zOEJSYmxsTEGBUWFxeT306bvv/++z///PPBgwdpuDYQ+t7RUQIYJYjJAABGgFKpXL9+fVlZWVdX14kTJ2JiYsRicXp6OlchJCSkoaFh27ZtnZ2dtbW1q1ev5m6hcR577LGamprLly+XlZXV1dUFBgbS8uDgYAcHh/Ly8tHrv1qtdnJyMnrUowk+Pj5G2Vap1NRUDw+PhISEoqIivV5fU1OzfPnyxsbG9PR0OoNpjtTUVE9Pz5UrVxYXF+t0ura2toyMjJSUlLS0NO5WVkxMDMMwFy9eNNFOdnZ2SkpKfX29wWCor69fu3ZtVlaWRqOJjY2lFfbu3fvBBx/8+OOPNjY2/HlSo8QchBCaiSMkJMTMSwAYCgH3fML4hVwY9yCCXBj/NLTPJ12czomOji4rK+OXbNiwgf3tzseFCxfS16rV6kmTJp09e1ar1drY2Mhksrlz5x47dozffnt7e2xsrIuLi0wmCwgIqKio0Gg0tJ21a9fSOufPnw8MDJTL5W5ubtu3b+deGxgYaG9vP7RMEKzZn43169dbWVldvXqV/tnc3My/WI1G0/8lr732mlEuDJZlW1paEhISPDw8RCKRUqnUarUlJSX0kPlDSpOcTZ06VSQSTZw4MSQk5PDhw/yzBAcHKxSK3t7egS5Hp9NlZmZqtVqa5EyhUGg0mtTU1Bs3bnB1+HsRjBhlHqELzm7dujXoMCIXBgwZw47m5mq4X0VGRpJ/JleEewTDMDk5OUuXLhW6I8Ib+8/nrFmzWlpahrYlcAyY+dnQ6XQ+Pj6hoaF3TAZxT2lvb3d1dY2Ojt69e/cYnK6qqsrX1zc7O3vZsmWDVs7NzY2KisJvKwwB5i5h/OFnTh/xxhUKBdOPVCqdOXMmvW8x5KbS0tKMKly5csXoRAUFBdzR5ORk/qGB0kENwaVLl1555RV3d3exWMy1v2nTppFq3wSWZUtLS19//XUvLy+JROLk5BQQEEATkHJ1fv311507dwYHB0+YMEEmkz3yyCPR0dHmz6nBkCmVysLCwry8vO3btwvdF1NYlo2Pj7e1tR0o09jIqquri4iISEpKMicgAxgOxGQw/iQmJrIsa+IhMMPR2dn5j3/8gxASHh5ObyYbDIby8nJbW9s33nhj7dq1Q24qMTHRqMLkyZNZlj1w4AD55+zVc889xx3dtGkTy7Jz586lCdC9vb1H4PIIaW5u9vPzO3XqVG5ubnt7O9tvOmlUVVdXBwQE1NTU5OXl6XS68vJyd3f3FStWrFmzhquzZs2aVatWhYeHnz17trW1dc+ePZWVlRqNhh+wwijx9fU9ceJEcXFxR0eH0H0Z0LVr1+rq6kpKSszcyDlMGRkZmzdv3rx58xicCx5wiMkABiEWi2fNmnXgwAELC4vPPvvMaBf9uJOZmdnU1PTZZ5/5+flZW1uP3okUCgX3eEc+Kyur3NzcmTNnSqXSqVOn7t2718HBYdu2bQaDgauzcuXK1atXq1Qqa2vrwMDA7Ozs27dvv/POO6PX2yGjd22rqqquXr3KMExycrLQPRquKVOmFBUV2draCt2RAalUqmPHjvn4+IzN6bZs2YI7ZDA2EJMBmMXNzc3FxaW3t3e8T6L99NNPhJAZM2YIcnZvb++enh57e3uuRCwWu7m5GQwGmkeeEJKZmWm0oU+tVstkstra2ntwjQ69a8sZmylgALgvISYDMBcNCIySZI47N27cIIQYpVkXUHt7+4ULF3x9fU08abGrq6u7u3v69OkCplQFABhtiMlgtBQUFHDrx6urq5cuXerg4ED/pE8abm5ujo+Pp9vUJ06cGBERQTMADcGmTZtoy9xk2XfffUdLHB0dR+Ryfvnll8bGRltbW/6MyQhewhig78jBgwcJITKZjD9c/dFMBJ6enmKx2N7efsGCBUeOHOGO9vb25uTkPP300yqVSiaTzZgxIz09nXvWEJ3O6+rqKi0tpe/CHbOxd3R0lJaWhoWFqVSqffv2meg53UG5YcOGoV04AMC4gJgMRstzzz3Hsmx4eDghJC4u7g9/+MPly5fLy8vp4+QaGxvnzJmTm5u7Y8eOtrY27hHLQ1tvnpyczLKsXC7nSp555hmWZbn8T5wh5N7s6emprKxcvny5SCTatm0bt85mZC9hDPDfke7ubpZljx07dseaTU1Nc+bMyc7OTk9Pb2lp+fHHH62trefPn5+ZmUkrfPfdd8uWLQsODj537tzly5f/4z/+46233uI2QNDpPLlc/uSTT9IZvf651zdt2qRUKgMCAiwtLfPz802kR7927dq6detiY2OR5gMA7m+IyWAsrF27NigoyNra+vHHH+/t7XV0dExKSrp06dKnn3767LPPKhQKHx+fr776imXZVatWjWpP+vr6aJQwaM2DBw/SezxisdjX19fJyens2bMrVqzgKgh1CWMgKSnp4sWLW7duDQ0NtbW19fLyys7OdnFxiY+Ppw8uJIQEBQUlJSXZ29s7OjquWrVq+fLl6enp5m/WS05ONhgM586d8/b29vX1HSipQWtr6zPPPBMUFHTvZ8wCABgmPIMcxsLvfvc7o5KCggILC4vQ0FCuRKVS+fj4nDx58sqVK6OReIw6evT/s3encVEd+f74q4HeoKFBtkaFiCjhitqSxqtM6KuCQ7uwjERCVMxkHDJMrhFI4oYaY4yGa8Ko3JcbkfGFBjEyZMARgxnD1XtHBAfNgHGFoBgRUBZpoMWW5fwf1D/nd9JsLSAH5PN+JHWq69SptPLNqapvnTOyZkhICE2+cP/+/Q8++OD48eMTJ07csWMHW2GgHoG+OGxvb+/yant7O60wmGhGe26Kc7FY7O/v/+WXX3777bdvvvlmYGAg98EJIUqlMjU19dq1az4+PkbeRSQSeXh47N+//8GDB5s3b/bx8Zk7dy63gk6n02g0kyZNOnLkyLMOQkVFRXp6+jN95MU2ZF/fvngw1NBniMlgMHBnFQkher1eq9USQrpc1l1aWvr8YrI+GDNmTEpKyqVLlz7//POwsDBvb28yoI8gk8kIId29YWpoaBjkrAT00SQSicE+AHpYYXV1NSFEq9X+6U9/yszMrKioaGhoYOvQDQTPKigoKDMzMzs7mxuTtbW10dNsDh8+3IeotKCgIDw8vA+deVHt3r179+7dfPcCAHqCuUvggVgstra2NjMza21t7Xzg15w5c/rWrImJydOnT7kl3HChPyQSyaeffsowzPr162nJAD6Cu7s7IeTatWudL+n1+h9//HHixIkD8hRGEovFcrn8yZMnTU1N3HI6a0mzdAYFBX3yySdvv/12SUkJnQ7etWsX+XlrKmX8HkmxWEwIMUj8FhUVpdfr09PT2f0BEyZMMH4hIM5j5SI4C3UQ0fMuAfoAMRnwIzQ0tK2tLS8vj1u4Y8cOFxeXzuvBjeTk5HT//n32x+rq6p9++qlfveQICwvz8vLKzc09c+YMLennI5iZmdHjktzc3Dw8PAoKCkpLSw3qpKen29vb97D+/TlZtGgRIeTUqVNsiV6vz83NlUqlGo2mvb09Ly9PoVBER0fb29vT2KulpcWgEXNzczZEfvnll7/44gtCyOrVqyMiIgxq5uTkEEKmT5/OlmzZsuXatWsnTpyg4RoAwEiAmAz4ER8f7+bmtmLFipycHK1WW19fn5SUtHXr1oSEhC7zJhgjICCgsrJyz549zc3NZWVlMTExDg4OBnX6sO+SYk+EXL9+PcMwA/sIu3btMjExmT9//l//+tf6+vr29vbKysp9+/a9++67O3fuNDEZ7L+n8fHxrq6usbGx2dnZTU1NJSUlS5curaqqSkxMdHR0NDU1nT17dnV19eeff15bW9vS0nL27NnOa/BfeeWVkpKSe/fu5efn3759W61W0/K0tLStW7eWl5fr9fry8vJ169alpqaqVKrIyEhaISUl5eOPP7548aKlpSX3xM+ysrJBHQUAgEHG91teGJYWL17c69xQ54WuBhVoBqzx48cLhUJ7e/uAgIAzZ84Yc/fPP/+c2+zGjRtpeUNDQ2RkpJOTk1Qq9fX1LSwsZHNh0NMkGYZRq9U2NjYXLlzornGDpW/h4eHcq2xCL5rloedHMGiqsxs3brCVL1++HBERMW7cOLFYLBKJxo4dGxYWlpeXZ8yAUKS3+Sm6cp+1bNkyhmHc3Ny4hffu3aOVa2trY2NjXV1dhUKhXC7XaDS5ublsUzU1NVFRUc7OzkKh0NHR8a233mJndVUqFa1z8+ZNtVptYWHh7OxMj29nGEar1SYnJ2s0GprUTSaTqVSq+Pj4x48fs41z9xYYyM/PN2YojPl+jii9fjdgANG5S757AcOSgBl6Z5XA0BcWFkZ+zuQJQ4RAIDh+/DiSeBF8PzvBd2Mwpaen0/+X47sjMPxg7hIAAACAf4jJAADg/7l7925wcHBjY2NtbS27mM/Ly4s9JJ7iXhUIBDRHzJDCMExeXt7KlSvd3d3FYrGDg4Ovr29qair3DdajR48OHDjg5+c3atQoqVQ6ceLEZcuWFRcXc9tZv349tlLC4EBMBkORoHtbtmzhu3cAL6yioiJvb++AgAArKys7OzuGYQoLC2l5bGwstya9mp+fb2tryzDMpUuXeOpyt27duuXr61tSUpKRkaHVagsKClxcXJYvX75mzRq2zpo1a1atWhUSEnL9+vW6urpDhw4VFRWpVCqaL5p6++234+LiPvzwQz4eAkYWxGQwFPWwBBIxGQw1Mpmsh9Pch377rMbGxqCgoNdee+3dd9/llovFYltb26SkpGPHjg1CNwaQmZlZenr61KlTJRLJ+PHjU1JSbG1t9+zZo9fr2TorVqyIiYlRKBTm5uZqtTotLa29vX3t2rVsBTc3t8zMzO3bt+NkCHjeEJMBAAAhhHz22WfV1dWbN282KJdIJEePHjUxMYmKiiopKeGlb33g4eHR2tpqY2PDlohEImdnZ71ez87DJicnJyUlcT+lVCqlUmlZWRl3ilOpVC5evPiDDz7oc/ZEAGMgJgMAAMIwTHJy8owZM0aPHt35qkaj2bRpU1NTU1hYmMHCsmGkoaGhtLTUy8uryyPRKJ1O19LSMnnyZIODKBYtWlRRUcFNpAww4BCTAcDIRTPMubm5iUQiGxub+fPnnz17ll7atm0bXcLIzhuePn2altjZ2dGShIQEgUCg0+ny8vLoJZoumJYLBIKxY8cWFhb6+/tbWlqam5vPmTOHPfihP+0/D8XFxQ8ePFAqld1V+OijjwICAq5cubJq1aqem+phVLOysti1oeXl5eHh4dbW1ra2toGBgQY5gWtqaqKjo2keO3t7LnoXNAAAIABJREFU+9DQ0KKioj4/XWNjY15eXnBwsEKhOHLkSA81aQqVjRs3GpRPmzaNEPLtt9/2uQ8AvXveCdDghYScnEMQQV7Qnxn5/ayqqnJ1dXV0dDx58qRWq71161ZoaKhAIDh48CBbx8LCguYHZqlUKrqqvYc6lFKptLCw8PHxuXDhQnNzc2Fh4dSpU0Ui0blz5wak/Tlz5owaNcqYPLrGfDe+/PJLQgg915WrsLBQLpfTP9fU1Dg7OxNC6O5FhrPGn2XMqIaEhBBCQkJC6MicOXNGKpVOnz6drVBZWfnSSy85OjqeOnWqqanp6tWrs2bNkkgkPWR77sEnn3xCf9/Nnj37ypUrPdSsrq52dHSMjIzsfEmr1RJC1Gp1r7dDzljoM7wnA4ARKi4u7s6dO7t37w4MDLSysnJ3d09LS3NycoqOjqYHrvefTqfbt2+fj4+PhYWFt7d3amrq06dPY2JiBqRxevo7M0C5SauqqgghPUzqEULs7OzS09OFQmFUVBQ9rbUz40c1MjKSjszcuXMXLlxYWFhYW1vLNnL37t2dO3cuWLBAJpN5enp+9dVXDMP0+oquS5s2bdLr9Tdu3PDw8PDy8mJDNAN1dXXz5s2bPXt254PCCCFWVlYCgYCOEsBzgpgMAEYoetgU9ygnsVjs7+/f0tIyUFNUFhYWdM6LmjJlyujRo4uLiwfkV/u5c+fq6+t9fHz63xQhhK4SEwqFPVebOXNmQkKCTqcLCwvrfPA8eZZR5Z46T1+/VVZW0h+zsrJMTEwCAwPZCgqFwtPT8/LlyxUVFc/6aIQQkUjk4eGxf//+4ODgzZs3f/fddwYVdDqdRqOZNGnS0aNHTU1Nu2zEzMysy0cGGCiIyQBgJNLr9VqtViKRWFpacssdHR0JIdXV1QNyF2tra4MSBwcHQsjDhw8HpP0BJJFICCGtra291oyOjg4PD7969apBygzyjKPKfScnEokIIR0dHWwjHR0dcrmcm5vw+++/J4SUlpb27QGpoKAgQkh2dja3sK2tLSwsbMyYMYcPH+4uIKPVpFJpf+4O0LPntVwUAGAoE4vFcrlcq9U2NTVxAwg6v6ZQKOiPJiYmT58+5X6woaHBoCmDDXpcdXV1DMNwK9BojEZm/W9/ADk5ORFC6KqpXiUnJxcVFR06dIhGciwjR7VnYrHY2tq6ubm5paVlwPc0iMViQkh9fT23MCoqSq/XZ2ZmsrebMGFCamrqzJkz2TqNjY0Mw9BRAnhO8J4MAEaoRYsWEUK42Q30en1ubq5UKtVoNLTEycnp/v37bIXq6uqffvrJoB1zc3M2rnr55Ze/+OIL9tKTJ09oHnzqhx9+qKysVCqV7K/2frY/gCZPnkwIMXJmUCaTff311xYWFvv27TO4ZMyo9io0NLStrY3do0rt2LHDxcXF+Axhq1evjoiIMCjMyckhv5w23bJly7Vr106cOEHDte7Q/0x0lACeE8RkADBCxcfHu7q6xsbGZmdnNzU1lZSULF26tKqqKjExkc61EUICAgIqKyv37NnT3NxcVlYWExPDvuJivfLKKyUlJffu3cvPz799+7ZarWYvyeXyDRs25Ofn63S6S5cuRUREiESixMREtkJ/2vfz87O1tS0oKBiQ0VAqlQ4ODgZHPfbA09PTINsqZcyo9io+Pt7NzW3FihU5OTlarba+vj4pKWnr1q0JCQnsq6yIiAiBQHDnzp0e2klLS9u6dWt5eblery8vL1+3bl1qaqpKpYqMjKQVUlJSPv7444sXL1paWnLnSQ0ScxBCaCaOgIAAIx8BoC943PMJwxdyYQxBBLkwfmb897O2tjY2NtbV1VUoFMrlco1Gk5uby63Q0NAQGRnp5OQklUp9fX0LCwtVKhX9x3PdunW0zs2bN9VqtYWFhbOz8969e9nPKpXKMWPGXL9+XaPRWFpaSqXSWbNmnT9/fqDaV6vVNjY2xqSHMPK7sWHDBjMzs/v379Mfa2pquL8sVCpV54+88847BrkwmB5HNT8/n9vmxo0bmV/uG124cCGtSZOcjR8/XigU2tvbBwQEnDlzhnsXPz8/mUzW1tbW3eNotdrk5GSNRkOTnMlkMpVKFR8f//jxY7YOdy+CAYMkI3TB2dOnT3sdRuTCgD4TMAO0jxpGlLCwMPJzckUYIgQCwfHjx19//XW+O8K/IfL9nDZtWm1tbd/2CQ4sI78bWq3W09MzMDCwy2QQQ0pDQ8Po0aOXLVt28ODBQbhdcXGxl5dXWlraG2+80Wvl9PT08PBw/G6FPsDcJQAAEEKIXC4/efJkRkbG3r17+e5LTxiGiY6OtrKy6i7T2MC6fft2aGhoXFycMQEZQH8gJgMAgP+fl5fXpUuXcnJyGhsb+e5Ltx48eHD79u3c3FwjN3L2U1JS0vbt27dv3z4I94IRDjEZAMAAo+dUFhcX379/XyAQbNq0ie8ePYNx48ZlZ2dbWVnx3ZFuKRSK8+fPe3p6Ds7tduzYgTdkMDiQnwwAYICtXr169erVfPcCAIYZvCcDAAAA4B9iMgAAAAD+ISYDAAAA4B9iMgAAAAD+YY0/9FFBQQHNzAlDx65du3hPlDoU0OOG8P3kwndj0AyFLMEwTCGPP/TFzp07Dc5IAeBRTk6Ol5fX4GSrAjAGImDoA8RkADDs4VwpAHgBYD0ZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwz4zvDgAAPLOGhgaGYbglOp3u0aNH7I8ymUwoFA56vwAA+k5g8O8aAMDQ5+fnd/bs2e6umpqa3r9/39HRcTC7BADQT5i7BIDhZ8mSJQKBoMtLJiYm//Ef/4GADACGHcRkADD8LF682Mys66UXAoHgzTffHOT+AAD0H2IyABh+bGxsAgICTE1NO18yMTFZtGjR4HcJAKCfEJMBwLAUERHR0dFhUGhmZrZw4UK5XM5LlwAA+gMxGQAMS8HBwWKx2KCwvb09IiKCl/4AAPQTYjIAGJbMzc0XLVpkkPBCKpUuWLCAry4BAPQHYjIAGK6WLl3a2trK/igUChcvXiyVSnnsEgBAnyEmA4DhSqPRcJeOtba2Ll26lMf+AAD0B2IyABiuhELhG2+8IRKJ6I/W1tb+/v78dgkAoM8QkwHAMLZkyZKnT58SQoRCYURERHdJywAAhj6crQQAw1hHR8fo0aMfPHhACDl//vyrr77Kd48AAPoI78kAYBgzMTFZvnw5IcTJyelXv/oV390BAOg7vOcf6fLz8+/du8d3LwD6zs7OjhAyY8aMv/zlL3z3BaBfXn/9db67AHzC3OVIFxYWlpGRwXcvAACA4DfyCIf3ZEAWL16MFwwjlkAgOH78+HD/v/OMjIzFixf3s5GwsDBCCP4usF6M78ZwkZ6eHh4ezncvgGdYTwYAw17/AzIAAN4hJgMAAADgH2IyAAAAAP4hJgMAAADgH2IyAAAAAP4hJgMAgP66e/ducHBwY2NjbW2t4GdeXl5PnjzhVuNeFQgE3t7efHW4OwzD5OXlrVy50t3dXSwWOzg4+Pr6pqamcrNUPHr06MCBA35+fqNGjZJKpRMnTly2bFlxcTG3nfXr1x8/fnzQuw/DG2IyAIB+aW5unjhxYmBgIN8d4U1RUZG3t3dAQICVlZWdnR3DMIWFhbQ8NjaWW5Nezc/Pt7W1ZRjm0qVLPHW5W7du3fL19S0pKcnIyNBqtQUFBS4uLsuXL1+zZg1bZ82aNatWrQoJCbl+/XpdXd2hQ4eKiopUKlVWVhZb5+23346Li/vwww/5eAgYrhCTAQD0C8MwHR0dHR0dfHVAJpP5+vrydffGxsagoKDXXnvt3Xff5ZaLxWJbW9ukpKRjx47x1be+MTMzS09Pnzp1qkQiGT9+fEpKiq2t7Z49e/R6PVtnxYoVMTExCoXC3NxcrVanpaW1t7evXbuWreDm5paZmbl9+/b09HQ+HgKGJcRkAAD9YmlpWVZW9s033/DdEX589tln1dXVmzdvNiiXSCRHjx41MTGJiooqKSnhpW994OHh0draamNjw5aIRCJnZ2e9Xs/OwyYnJyclJXE/pVQqpVJpWVkZd4pTqVQuXrz4gw8+aGtrG5zOw3CHmAwAAPqIYZjk5OQZM2aMHj2681WNRrNp06ampqawsDCDhWXDSENDQ2lpqZeXl1wu766OTqdraWmZPHmyQCDgli9atKiiouLUqVPPv5vwIkBMBgDQd1lZWeyKdRp2cEvKy8vDw8Otra1tbW0DAwPLysropxISEmiFsWPHFhYW+vv7W1pampubz5kzJy8vj9bZtm0brcPOS54+fZqW0GPX2XZ0Ol1eXh69ZGY2qCfmFRcXP3jwQKlUdlfho48+CggIuHLlyqpVq3puqq6u7v3333dzcxOJRDY2NvPnzz979iy9ZMyQUjU1NdHR0ePGjROJRPb29qGhoUVFRX1+usbGxry8vODgYIVCceTIkR5q0iO5Nm7caFA+bdo0Qsi3337b5z7AyMLAyLZ48eLFixfz3QvgDSHk+PHjfPdiSOjP34WQkBBCSEtLi0FJSEjIhQsXmpubz5w5I5VKp0+fzv2UUqm0sLDw8fGhdQoLC6dOnSoSic6dO8fWsbCwePXVV7mfUqlUdIF8D3WoOXPmjBo1Kj8/v28PZcx348svvySEfPrppwblhYWFcrmc/rmmpsbZ2ZkQQncvMpw1/qyqqipXV1dHR8eTJ09qtdpbt26FhoYKBIKDBw+ydXod0srKypdeesnR0fHUqVNNTU1Xr16dNWuWRCK5cOFCHx7/k08+ob8lZ8+efeXKlR5qVldXOzo6RkZGdr6k1WoJIWq1utfb0U2afegnvEjwngwA4HmJjIz08fGxsLCYO3fuwoULCwsLa2truRV0Ot2+fftoHW9v79TU1KdPn8bExAzI3Ts6Oug/9APSWpeqqqoIIT1M6hFC7Ozs0tPThUJhVFTUzZs3u6wTFxd3586d3bt3BwYGWllZubu7p6WlOTk5RUdHP3jwgFuzhyGNi4u7e/fuzp07FyxYIJPJPD09v/rqK4Zhen1F16VNmzbp9fobN254eHh4eXmxIZqBurq6efPmzZ49+8CBA52vWllZCQQCOkoAvUJMBgDwvEyfPp39M31XVFlZya1gYWFBp7eoKVOmjB49uri4eEB+i587d66+vt7Hx6f/TXWHTtcKhcKeq82cOTMhIUGn04WFhbW0tHSukJmZSQhZuHAhWyIWi/39/VtaWgwm/noY0qysLBMTE25SEoVC4enpefny5YqKimd9NEKISCTy8PDYv39/cHDw5s2bv/vuO4MKOp1Oo9FMmjTp6NGjpqamXTZiZmbW5SMDdIaYDADgeeG+QBKJRIQQg5QZ1tbWBh9xcHAghDx8+PD5924ASCQSQkhra2uvNaOjo8PDw69evWqQMoMQotfrtVqtRCKxtLTkljs6OhJCqquruYXdDSltpKOjQy6Xc9PSfv/994SQ0tLSvj0gFRQURAjJzs7mFra1tYWFhY0ZM+bw4cPdBWS0mlQq7c/dYeQY1NWgAADAVVdXxzAMd7MejcZoZEYIMTExefr0KfcjDQ0NBo0Y7PUbTE5OToQQumqqV8nJyUVFRYcOHaKRHEssFsvlcq1W29TUxA3L6KylQqEwpnGxWGxtbd3c3NzS0jLgGx3EYjEhpL6+nlsYFRWl1+szMzPZ202YMCE1NXXmzJlsncbGRoZh6CgB9ArvyQAAePPkyROa8p764YcfKisrlUol+1vcycnp/v37bIXq6uqffvrJoBFzc3M2bnv55Ze/+OKL59zr/2fy5MmEECNnBmUy2ddff21hYbFv3z6DS4sWLSKEcHNG6PX63NxcqVSq0WiM7ExoaGhbWxu7cZXasWOHi4uL8RnCVq9eHRERYVCYk5NDfjltumXLlmvXrp04cYKGa92h/+3oKAH0CjEZAABv5HL5hg0b8vPzdTrdpUuXIiIiRCJRYmIiWyEgIKCysnLPnj3Nzc1lZWUxMTHsKzTWK6+8UlJScu/evfz8/Nu3b6vValru5+dna2tbUFDw/PqvVCodHBwMjnrsgaenp0G2VSo+Pt7V1TU2NjY7O7upqamkpGTp0qVVVVWJiYl0BtMY8fHxbm5uK1asyMnJ0Wq19fX1SUlJW7duTUhIYF9lRURECASCO3fu9NBOWlra1q1by8vL9Xp9eXn5unXrUlNTVSpVZGQkrZCSkvLxxx9fvHjR0tKSO09qkJiDEEIzcQQEBBj5CDDS8bjnE4YC5MIY4QhyYfysb38X6OJ01rJly/Lz87klGzduZH6583HhwoX0s0qlcsyYMdevX9doNJaWllKpdNasWefPn+e239DQEBkZ6eTkJJVKfX19CwsLVSoVbWfdunW0zs2bN9VqtYWFhbOz8969e9nPqtVqGxubvmWCYIz+bmzYsMHMzOz+/fv0x5qaGu7DqlSqzh955513DHJhMAxTW1sbGxvr6uoqFArlcrlGo8nNzaWXjB9SmuRs/PjxQqHQ3t4+ICDgzJkz3Lv4+fnJZLK2trbuHker1SYnJ2s0GprkTCaTqVSq+Pj4x48fs3W4exEMGGQeoQvOnj592uswIhcGMAwjYJ7nNmkY+sLCwsjPCQ9hBBIIBMePH3/99df57gj/Bv/vwrRp02pra/u2JXAQGPnd0Gq1np6egYGBXSaDGFIaGhpGjx69bNmygwcPDsLtiouLvby80tLS3njjjV4rp6enh4eH4zfyCIe5S+iLr776ir6rN1irO1x888037u7ufV4ILJPJuBMWJiYmNjY2SqXyP//zPy9fvjywXR067t69+7vf/c7FxUUkErHPvm3btkG4NcMweXl5K1eudHd3F4vFDg4Ovr6+NAEpW+fRo0cHDhzw8/MbNWqUVCqdOHHismXLjJ9Tgz6Ty+UnT57MyMjYu3cv333pCcMw0dHRVlZW3WUaG1i3b98ODQ2Ni4szJiADoBCTQV+88cYbDMP4+/vz3ZFnVlZWFhwcHBcXZ5CI8pk0Nzf/61//IoSEhIQwDNPa2nrz5s2tW7fevHnT29v7d7/73ePHjweuy0NCTU3NzJkzv//++/T09IaGBqbTdNJzdevWLV9f35KSkoyMDK1WW1BQ4OLisnz58jVr1rB11qxZs2rVqpCQkOvXr9fV1R06dKioqEilUmVlZQ1aP0csLy+vS5cu5eTkNDY28t2Xbj148OD27du5ublGbuTsp6SkpO3bt2/fvn0Q7gUvDMRkMLJ8+OGHv/rVry5fvmyQCak/TE1NHR0dQ0JC/ud//mft2rUpKSlLlix5weYgkpOTq6urd+3aNXPmTHNz8+d3I5lMxh7vyGVmZpaenj516lSJRDJ+/PiUlBRbW9s9e/bo9Xq2zooVK2JiYhQKhbm5uVqtTktLa29vX7t27fPrbZ/RcyqLi4vv378vEAg2bdrEd4/6a9y4cdnZ2VZWVnx3pFsKheL8+fOenp6Dc7sdO3bgDRk8K+Qng5Hlz3/+83PN3/hf//Vf//u///u3v/3tq6++WrJkyfO70SD74YcfCCFTpkzh5e4eHh4GWUlFIpGzs3NRUdGTJ09oMoLk5GSDTymVSqlUWlZWxvwyAdhQsHr16tWrV/PdCwAYWvCeDEaW551QWyAQ0DTlnTMwDWt0NnYAXy72U0NDQ2lpqZeXVw8nLep0upaWlsmTJw+1gAwAoEuIycBYN2/e/M1vfiOXyy0sLNRq9fnz5zvXqampiY6OpnvI7e3tQ0NDaXoeQkhWVha7MLy8vDw8PNza2trW1jYwMJCb1Eev12/evNnDw8Pc3HzUqFFBQUF/+9vf2tvbjbnFEEGn3goKCthXO8N6WGgPT5w4QQiRSqUCgaDLuUWKZiJwc3MTiUQ2Njbz588/e/Yse7Wtre348eO//vWvFQqFVCqdMmVKYmIie9YQnc7T6XR5eXl0QLrchNHY2JiXlxccHKxQKI4cOdJDz+kOyo0bN/btwQEABhtfSThgiDAyJ1Npaam1tfWYMWP+/ve/NzU1XblyJSAgYNy4cWKxmK1TWVn50ksvOTo6njp1qqmp6erVq7NmzZJIJNz0SCEhIYSQkJCQCxcuNDc3nzlzRiqVTp8+na0QGRkpl8v//ve/P378uLq6ms7vnD171vhbGGnMmDGmpqZdXpozZ86oUaMM8gwZ4K7xN8CeN1xZWTn0h4UYl4OK9rClpYUtoWv8P/nkE7akqqrK1dXV0dHx5MmTWq321q1boaGhAoHg4MGDtMLJkycJIZ9++ml9fX1NTc1///d/m5iYrF69mnsjCwuLV199tbtusDvmZs+efeXKlR46XF1d7ejoGBkZ2eujsZCrz4CR3w0YEMhPBgzD4Bsw0hn5e4imbsrIyGBL7t+/LxaLuTHZb3/7W0LI0aNH2ZKqqiqxWMxNGkl/tZ88eZLbAUJITU0N/dHV1fVXv/oV99bu7u5s8GHMLYzUQ0w2a9asXjNt9hCTsZsuaUw2xIdlAGOyt956ixBy7NgxtuTJkyejR4+WSqXV1dUMw5w8eXL27NncZiMiIoRCoVarZUt6jskYhtHr9Tdu3PjjH/9oamq6devWLuvU1tZOmzYtPDy8h9SgnSEmM4CYbDAhJgOGYbDGH4xy+vRpQgj34LnRo0e7u7uXlJSwJVlZWSYmJoGBgWyJQqHw9PS8fPlyRUXF2LFj2XLusXHOzs6EkMrKSjs7O0LIvHnz9u/f/4c//GHFihXTp083NTW9detWH27RH+fOnevPx6uqqgghQqGQPtELMyy9ohntuSnOxWKxv7//l19++e2337755puBgYHcThJClEplamrqtWvXfHx8jLyLSCTy8PDYv3//gwcPNm/e7OPjM3fuXG4FnU6n0WgmTZp05MgRU1PTZ3qEgoIC+r8fQO3atQsJpQfHkE0dDIMJ68mgd3q9vqmpSSKRyGQybjn33D29Xq/Vajs6OuRyOTef6vfff08IKS0t5X6Quy5bJBIRQthFRXv37j1y5Mjt27f9/f2trKzmzZvHnl3zTLfgEV1p5+PjIxQKR86w0G5IJBKDfQD0sMLq6mpCiFar3bx585QpU2xsbGgnaYKxvqVzCwoKIoRkZ2dzC9va2uhpNocPH37WgAwAgF94Twa9E4vFlpaWTU1Nzc3N3LCsvr6eW8fa2rq5ubmlpaXP+fEJIQKBYPny5cuXL29tbT137lxCQkJoaOif/vSn999/f6Bu8Vx1dHTQbOYrV64kI2lYxGKxXC7XarVNTU3csIzm5qVZOoOCgv7xj38kJiYuWbLEzs5OIBDs3r37vffeYzi53IzfI0lTYHC/hISQqKgovV6fmZnJDsWECRNSU1NnzpxpTJszZ87EayGWQCB47733cO7W4KBnK/HdC+AZ3pOBUebPn09+nsGkamtrudNnhJDQ0NC2tra8vDxu4Y4dO1xcXNra2oy8kbW19c2bNwkhQqHw17/+Nd30d+rUqQG8xXMVFxf3z3/+c9GiRewU2MgZlkWLFhFC2F4RQvR6fW5urlQq1Wg07e3teXl5CoUiOjra3t6exl7sfgiWubn506dP6Z9ffvnlL774ghCyevXqiIgIg5o5OTnklxO+W7ZsuXbt2okTJ2i4BgAwvCAmA6N8+umno0aNio2NPXPmTHNz8/Xr1yMiIgymMuPj493c3FasWJGTk6PVauvr65OSkrZu3ZqQkPBM72/++Mc/XrlyRa/XP3z48LPPPmMYxs/Pb2Bv0TM/Pz9bW9uCggIj63d0dDx8+PDEiRP+/v6fffbZihUrjh49yr7veWGGpVfx8fGurq6xsbHZ2dlNTU0lJSVLly6tqqpKTEx0dHQ0NTWdPXt2dXX1559/Xltb29LScvbs2c6nVr/yyislJSX37t3Lz8+/ffu2Wq2m5WlpaVu3bi0vL9fr9eXl5evWrUtNTVWpVJGRkbRCSkrKxx9/fPHiRUtLS+4cLjelCADAkMbzHgPgm/F7zW7duvWb3/zGysqKpmnIzs5mz7v8/e9/T+vQ9FTjx48XCoX29vYBAQFnzpyhlwyOR9y4cSPzy9OHFi5cyDBMUVFRVFTUv/3bv9FEXDNnzjx48GBHRwfbjR5uYQyajsEAm6yBUqvVPe+7tLCw4H5cIBDI5fIpU6a88847ly9f7lx/KA8L6W1vHbtwjVq2bBnDMG5ubtzCe/fu0cq1tbWxsbGurq5CoVAul2s0mtzcXLapmpqaqKgoZ2dnoVDo6Oj41ltvrV+/nrbAbhG9efOmWq22sLBwdnbeu3cvLdRqtcnJyRqNhiZgk8lkKpUqPj7+8ePHbOPcvQUGek5rwsK+SwO9fjdgAGHfJTAMI2BerFP54FnRKTasoRmxBALB8ePHsWaI4O9CJ/huDCa6ngy/kUc4zF0CAEB/3b17Nzg4uLGxsba2lp049vLyevLkCbca96pAIPD29uarw91hGCYvL2/lypXu7u5isdjBwcHX1zc1NZUbLT169OjAgQN+fn6jRo2SSqUTJ05ctmxZcXExt53169fTV18AxkNMBgAA/VJUVOTt7R0QEGBlZWVnZ8cwTGFhIS2PjY3l1qRX8/PzbW1tGYa5dOkST13u1q1bt3x9fUtKSjIyMrRabUFBgYuLy/Lly2neFmrNmjWrVq0KCQm5fv16XV3doUOHioqKVCpVVlYWW+ftt9+Oi4v78MMP+XgIGK4Qk8GLQ9C9LVuT+eizAAAgAElEQVS28N07gF+QyWQ9nBw69NtnNTY2BgUFvfbaa++++y63XCwW29raJiUlHTt2bBC6MYDMzMzS09OnTp0qkUjGjx+fkpJia2u7Z88evV7P1lmxYkVMTIxCoTA3N1er1Wlpae3t7WvXrmUruLm5ZWZmbt++PT09nY+HgGFpiCZ5AugDLMUAGHyfffZZdXX15s2bDcolEsnRo0cXLFgQFRWlUqnc3d156d6z8vDwaG1t5ZaIRCJnZ+eioqInT57QNCvJyckGn1IqlVKptKysjGEYds+1UqlcvHjxBx98EBoaOmRTKsKQgvdkAADQRwzDJCcnz5gxY/To0Z2vajSaTZs2NTU1hYWFGSwsG0YaGhpKS0u9vLy4J20Y0Ol0LS0tkydPNkh6vGjRooqKCm7SPoAeICYDAHg2NPOIm5ubSCSysbGZP3/+2bNn6aVt27bR6XJ23vD06dO0hB5dSghJSEgQCAQ6nS4vL49eoi9RaLlAIBg7dmxhYaG/v7+lpaW5ufmcOXPYhMD9af95KC4ufvDggVKp7K7CRx99FBAQcOXKlVWrVvXcVA+jSpMkU+Xl5eHh4dbW1ra2toGBgQb552pqaqKjo2nOFHt7+9DQ0KKioj4/XWNjY15eXnBwsEKhOHLkSA816XbdjRs3GpRPmzaNEPLtt9/2uQ8wsvCVhAOGCORkGuEIclD9zMi/C1VVVa6uro6OjidPntRqtbdu3QoNDRUIBNwsdxYWFq+++ir3UyqViq5q76EOpVQqLSwsfHx8Lly40NzcXFhYOHXqVJFIdO7cuQFpf86cOaNGjTImZ5sx340vv/ySEPLpp58alBcWFsrlcvrnmpoaZ2dnQgjdvchw1vizjBnVkJAQQkhISAgdmTNnztBciWyFysrKl156ydHR8dSpU01NTVevXp01a5ZEIukh12APPvnkE/pbcvbs2VeuXOmhZnV1taOjY2RkZOdLWq2WEKJWq3u9HfKTAcMweE8GAPAM4uLi7ty5s3v37sDAQCsrK3d397S0NCcnp+joaHq4Z//pdLp9+/b5+PhYWFh4e3unpqY+ffo0JiZmQBpnUw0PSGtVVVWEkB4m9QghdnZ26enpQqEwKiqKHhHWmfGjGhkZSUdm7ty5CxcuLCwsrK2tZRu5e/fuzp07FyxYIJPJPD09v/rqK4Zhen1F16VNmzbp9fobN254eHh4eXmxIZqBurq6efPmzZ49u/OhFIQQKysrgUBARwmgV4jJAACeAT3YgHtsgFgs9vf3b2lpGagpKgsLCzrnRU2ZMmX06NHFxcUD8qv93Llz9fX1Pj4+/W+KEEJXiQmFwp6rzZw5MyEhQafThYWFdT7klDzLqHJPOKWv3yorK+mPWVlZJiYmgYGBbAWFQuHp6Xn58uWKiopnfTRCiEgk8vDw2L9/f3Bw8ObNm7/77juDCjqdTqPRTJo06ejRo6ampl02YmZm1uUjA3SGmAwAwFh6vV6r1UokEktLS265o6MjIaS6unpA7mJtbW1Q4uDgQAh5+PDhgLQ/gCQSCSHEYKNil6Kjo8PDw69evWqQMoM846hy38mJRCJCSEdHB9tIR0eHXC7n5sH5/vvvCSGlpaV9e0AqKCiIEJKdnc0tbGtrCwsLGzNmzOHDh7sLyGg1qVTan7vDyIHduQAAxhKLxXK5XKvVNjU1cQMIOr+mUCjojyYmJk+fPuV+sKGhwaApgw16XHV1dQwnpQL5ORqjkVn/2x9ATk5OhBC6aqpXycnJRUVFhw4dopEcy8hR7ZlYLLa2tm5ubm5paRnwPQ00BUZ9fT23MCoqSq/XZ2ZmsrebMGFCamrqzJkz2TqNjY0Mw9BRAugV3pMBADyDRYsWEUK42Q30en1ubq5UKtVoNLTEycnp/v37bIXq6uqffvrJoB1zc3M2rnr55Ze/+OIL9tKTJ09oHnzqhx9+qKysVCqV7K/2frY/gCZPnkwIMXJmUCaTff311xYWFvv27TO4ZMyo9io0NLStrY3do0rt2LHDxcWlra3NyEZWr14dERFhUJiTk0N+OW26ZcuWa9eunThxgoZr3aH/megoAfQKMRkAwDOIj493dXWNjY3Nzs5uamoqKSlZunRpVVVVYmIinWsjhAQEBFRWVu7Zs6e5ubmsrCwmJoZ9xcV65ZVXSkpK7t27l5+ff/v2bbVazV6Sy+UbNmzIz8/X6XSXLl2KiIgQiUSJiYlshf607+fnZ2trW1BQMCCjoVQqHRwcDI567IGnp2dSUlLncmNGtVfx8fFubm4rVqzIycnRarX19fVJSUlbt25NSEhgX2VFREQIBII7d+700E5aWtrWrVvLy8v1en15efm6detSU1NVKlVkZCStkJKS8vHHH1+8eNHS0pI7T2qQmIMQQjNxBAQEGPkIMNLxuOcThgLkwhjhCHJh/Mz4vwu1tbWxsbGurq5CoVAul2s0mtzcXG6FhoaGyMhIJycnqVTq6+tbWFioUqnoP7nr1q2jdW7evKlWqy0sLJydnffu3ct+VqlUjhkz5vr16xqNxtLSUiqVzpo16/z58wPVvlqttrGxMSY9hJHfjQ0bNpiZmd2/f5/+WFNTw/0Vo1KpOn/knXfeMciFwfQ4qvn5+dw2N27cyPxy3+jChQtpTZrkbPz48UKh0N7ePiAg4MyZM9y7+Pn5yWSytra27h5Hq9UmJydrNBqa5Ewmk6lUqvj4+MePH7N1uHsRDBgkGaELzp4+fdrrMCIXBjAMI2BwHM3IFhYWRn5OeAgjkEAgOH78+Ouvv853R/g3RP4uTJs2rba2tm/7BAeWkd8NrVbr6ekZGBjYZTKIIaWhoWH06NHLli07ePDgINyuuLjYy8srLS3tjTfe6LVyenp6eHg4fiOPcJi7BACAvpPL5SdPnszIyNi7dy/ffekJwzDR0dFWVlbdZRobWLdv3w4NDY2LizMmIAOgEJMBAEC/eHl5Xbp0KScnp7Gxke++dOvBgwe3b9/Ozc01ciNnPyUlJW3fvn379u2DcC94YSAmAwAYEug5lcXFxffv3xcIBJs2beK7R89g3Lhx2dnZVlZWfHekWwqF4vz5856enoNzux07duANGTwr5CcDABgSVq9evXr1ar57AQC8wXsyAAAAAP4hJgMAAADgH2IyAAAAAP4hJgMAAADgH2IyAAAAAP4hj/9IFxYWlpGRwXcvAACA4DfyCIeYbKTLz8+/d+8e370A6Jfw8PDY2FgfHx++OwLQLzjlbIRDTAYAwx5O7QSAFwDWkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAAADwDzEZAAAAAP/M+O4AAMAzO3bsWFNTE7fku+++a2hoYH9ctGiRvb39oPcLAKDvBAzD8N0HAIBn89Zbbx0+fFgoFNIf6b9jAoGAENLe3i6TyR4+fCgWi/nsIgDAM8LcJQAMP0uWLCGEtP6sra2tra2N/tnU1DQsLAwBGQAMO3hPBgDDT1tbm6OjY319fZdXc3Nz/fz8BrlLAAD9hPdkADD8mJmZLVmyhJ275LKzs5s1a9bgdwkAoJ8QkwHAsLRkyZLW1laDQqFQuHz5clNTU166BADQH5i7BIBhiWEYFxeXiooKg/J//vOf06dP56VLAAD9gfdkADAsCQSCiIgIg+lLZ2dnb29vvroEANAfiMkAYLgymL4UCoVvvfUWzYgBADDsYO4SAIYxDw+PW7dusT9evXrV09OTx/4AAPQZ3pMBwDC2fPlydvpy0qRJCMgAYPhCTAYAw1hERERbWxshRCgU/va3v+W7OwAAfYe5SwAY3ry9vS9fviwQCMrLy11cXPjuDgBAH+E9GQAMb2+++SYhZMaMGQjIAGBYM+O7AzAUhYWF8d0FAGM9efJEIBDo9Xp8b2EYef/99318fPjuBQwteE8GXcjIyOicihOGu4KCgoKCAr57MfAkEomjo+PYsWON/0hFRUVGRsbz69Kw86J+N4asjIyMe/fu8d0LGHLwngy69t57773++ut89wIGEn2N9Je//IXvjgy8H3/8ccKECcbXT09PDw8PfyGHom9e4O/G0IQsetAlvCcDgGHvmQIyAIChCTEZAAAAAP8QkwEAAADwDzEZAAAAAP8QkwEAwLO5e/ducHBwY2NjbW2t4GdeXl5PnjzhVuNeFQgE3t7efHW4OwzD5OXlrVy50t3dXSwWOzg4+Pr6pqamcrOpP3r06MCBA35+fqNGjZJKpRMnTly2bFlxcTG3nfXr1x8/fnzQuw8vGsRkAADGam5unjhxYmBgIN8d4VNRUZG3t3dAQICVlZWdnR3DMIWFhbQ8NjaWW5Nezc/Pt7W1ZRjm0qVLPHW5W7du3fL19S0pKcnIyNBqtQUFBS4uLsuXL1+zZg1bZ82aNatWrQoJCbl+/XpdXd2hQ4eKiopUKlVWVhZb5+23346Li/vwww/5eAh4cSAmAwAwFsMwHR0dHR0dfHVAJpP5+vrydXdCSGNjY1BQ0Guvvfbuu+9yy8Visa2tbVJS0rFjx/jqW9+YmZmlp6dPnTpVIpGMHz8+JSXF1tZ2z549er2erbNixYqYmBiFQmFubq5Wq9PS0trb29euXctWcHNzy8zM3L59e3p6Oh8PAS8IxGQAAMaytLQsKyv75ptv+O4Ibz777LPq6urNmzcblEskkqNHj5qYmERFRZWUlPDStz7w8PBobW21sbFhS0QikbOzs16vZ+dhk5OTk5KSuJ9SKpVSqbSsrIw7xalUKhcvXvzBBx+0tbUNTufhxYOYDAAAjMIwTHJy8owZM0aPHt35qkaj2bRpU1NTU1hYmMHCsmGkoaGhtLTUy8tLLpd3V0en07W0tEyePNkg9euiRYsqKipOnTr1/LsJLybEZAAARsnKymKXq9OYg1tSXl4eHh5ubW1ta2sbGBhYVlZGP5WQkEArjB07trCw0N/f39LS0tzcfM6cOXl5ebTOtm3baB12XvL06dO0xM7OjtuOTqfLy8ujl8zMBvsgluLi4gcPHiiVyu4qfPTRRwEBAVeuXFm1alXPTdXV1b3//vtubm4ikcjGxmb+/Plnz56ll4wZVaqmpiY6OnrcuHEikcje3j40NLSoqKjPT9fY2JiXlxccHKxQKI4cOdJDTXrgwcaNGw3Kp02bRgj59ttv+9wHGOkYgE4IIcePH+e7FzDAFi9evHjxYr57MSTQLXJ9+2xISAghpKWlxaAkJCTkwoULzc3NZ86ckUql06dP535KqVRaWFj4+PjQOoWFhVOnThWJROfOnWPrWFhYvPrqq9xPqVQqujq+hzrUnDlzRo0alZ+f37eHMvK78eWXXxJCPv30U4PywsJCuVxO/1xTU+Ps7EwIobsXGc4af1ZVVZWrq6ujo+PJkye1Wu2tW7dCQ0MFAsHBgwfZOr2OamVl5UsvveTo6Hjq1KmmpqarV6/OmjVLIpFcuHChDyPwySef0N+Js2fPvnLlSg81q6urHR0dIyMjO1/SarWEELVa3evt8G8sdAnvyQAABkBkZKSPj4+FhcXcuXMXLlxYWFhYW1vLraDT6fbt20freHt7p6amPn36NCYmZkDu3tHRQf9NH5DWulNVVUUI6WFSjxBiZ2eXnp4uFAqjoqJu3rzZZZ24uLg7d+7s3r07MDDQysrK3d09LS3NyckpOjr6wYMH3Jo9jGpcXNzdu3d37ty5YMECmUzm6en51VdfMQzT6yu6Lm3atEmv19+4ccPDw8PLy4sN0QzU1dXNmzdv9uzZBw4c6HzVyspKIBDQUQLoA8RkAAADYPr06eyf6YuiyspKbgULCws6t0VNmTJl9OjRxcXFA/Ir/Ny5c/X19T4+Pv1vqgd0xlYoFPZcbebMmQkJCTqdLiwsrKWlpXOFzMxMQsjChQvZErFY7O/v39LSYjDx18OoZmVlmZiYcPOSKBQKT0/Py5cvV1RUPOujEUJEIpGHh8f+/fuDg4M3b9783XffGVTQ6XQajWbSpElHjx41NTXtshEzM7MuHxnAGIjJAAAGAPftkUgkIoQYpMywtrY2+IiDgwMh5OHDh8+/dwNDIpEQQlpbW3utGR0dHR4efvXqVYOUGYQQvV6v1WolEomlpSW33NHRkRBSXV3NLexuVGkjHR0dcrmcm5b2+++/J4SUlpb27QGpoKAgQkh2dja3sK2tLSwsbMyYMYcPH+4uIKPVpFJpf+4OI9lgLxEFABiZ6urqGIbh7tSj0RiNzAghJiYmT58+5X6koaHBoBGDjX6DzMnJiRBCV031Kjk5uaio6NChQzSSY4nFYrlcrtVqm5qauGEZnbVUKBTGNC4Wi62trZubm1taWgZ8r4NYLCaE1NfXcwujoqL0en1mZiZ7uwkTJqSmps6cOZOt09jYyDAMHSWAPsB7MgCAwfDkyROa75764YcfKisrlUol+yvcycnp/v37bIXq6uqffvrJoBFzc3M2bnv55Ze/+OKL59zrX5g8eTIhxMiZQZlM9vXXX1tYWOzbt8/g0qJFiwgh3JwRer0+NzdXKpVqNBojOxMaGtrW1sbuXaV27Njh4uJifIaw1atXR0REGBTm5OSQX06bbtmy5dq1aydOnKDhWnfofz46SgB9gJgMAGAwyOXyDRs25Ofn63S6S5cuRUREiESixMREtkJAQEBlZeWePXuam5vLyspiYmLYV2isV155paSk5N69e/n5+bdv31ar1bTcz8/P1ta2oKDguT6CUql0cHAwOOqxB56engbZVqn4+HhXV9fY2Njs7OympqaSkpKlS5dWVVUlJibSGUxjxMfHu7m5rVixIicnR6vV1tfXJyUlbd26NSEhgX2VFRERIRAI7ty500M7aWlpW7duLS8v1+v15eXl69atS01NValUkZGRtEJKSsrHH3988eJFS0tL7jypQWIOQgjNxBEQEGDkIwAY4nHPJwxZBPu0X0TIhcHqWy4MujKdtWzZsvz8fG7Jxo0bmV/ufFy4cCH9rFKpHDNmzPXr1zUajaWlpVQqnTVr1vnz57ntNzQ0REZGOjk5SaVSX1/fwsJClUpF21m3bh2tc/PmTbVabWFh4ezsvHfvXvazarXaxsamb2kgmGf5bmzYsMHMzOz+/fv0x5qaGu7zqlSqzh955513DHJhMAxTW1sbGxvr6uoqFArlcrlGo8nNzaWXjB9VmuRs/PjxQqHQ3t4+ICDgzJkz3Lv4+fnJZLK2trbuHker1SYnJ2s0GprkTCaTqVSq+Pj4x48fs3W4exEMGCQfoQvOnj592usw4t9Y6JKAec57p2E4EggEx48ff/311/nuCAyksLAw8nO6yxEuPT09PDx8MP/1mzZtWm1tbd/2Aw4C478bWq3W09MzMDCwy2QQQ0pDQ8Po0aOXLVt28ODBQbhdcXGxl5dXWlraG2+80Wtl/BsLXcLcJQAAGEsul588eTIjI2Pv3r1896UnDMNER0dbWVl1l2lsYN2+fTs0NDQuLs6YgAygO4jJYGjhHkTzPNr/5ptv3N3d+7xRSyaTCTqRSCRTp06lE0l9biohIcGgQkVFhcGNsrKy2KubNm3iXuouOWef3b1793e/+52Li4tIJGLvsm3btoG9S5cYhsnLy1u5cqW7u7tYLHZwcPD19aVJ4dk6jx49OnDggJ+f36hRo6RS6cSJE5ctW2b8OifoDy8vr0uXLuXk5DQ2NvLdl249ePDg9u3bubm5Rm7k7KekpKTt27dv3759EO4FLzI+J05hqCJ8r3Wgi28Gts0ff/wxKCho6tSpVlZWpqamfW7nX//6FyEkJCSE/qjX6//1r3+9+uqrhJA1a9b0p6kuHTt2jHCWExmYNWsW9zianhm/Zujhw4cKhWLq1Kl0QTrz8xKfTz75xMh79ceNGzcIIXPnzi0uLm5paSkrK1uyZAkh5IMPPmDr/P73vzczM9u9e3dVVZVOp/u///u/SZMmmZqaZmZmGnOL/pyt9Kw+//xz7j+5dHXUUIO1hoOM939jYWjCezIYKT788MNf/epXly9fNshU2U8ikWjatGnHjh0zMTHZtWuXQU6jYSo5Obm6unrXrl0zZ840Nzd/fjeSyWTsqdtcZmZm6enpU6dOlUgk48ePT0lJsbW13bNnj16vZ+usWLEiJiZGoVCYm5ur1eq0tLT29va1a9c+v972zerVq7n/5g7Ou0YAGI6QMxZGij//+c/PL7+2s7MzTS5VXFw8Z86c53SXQfPDDz8QQqZMmcLL3T08PAwyxYtEImdn56KioidPntAEUcnJyQafUiqVUqm0rKyM+WVeVgCA4QLvyWCkeN4HnjAMQ34+fGa4e/z4MSFkYF8o9kdDQ0NpaamXl1cPp1/rdLqWlpbJkycjIAOAYQoxGfRFVlYWu+771q1br7/+uq2tLf2xtraWEFJTUxMdHU1T/tjb24eGhtJsin2wbds22jI7yXX69GlaYmdnN2CP1D8//fRTVVWVlZWVp6cnWziAgzBo6H/ZEydOEEKkUil32Duj2aHc3NxEIpGNjc38+fPPnj3LXm1razt+/Pivf/1rhUIhlUqnTJmSmJjIHgFJN3PodLq8vDz6X7PLjReNjY15eXnBwcEKheLIkSM99Jzmcdi4cWPfHhwAgHeIyaAvfvOb3zAMExISQgiJior6z//8z3v37hUUFNCjeauqqqZPn56enr5v3776+vpz587V19f7+PgYpII00qZNmxiGsbCwYEvmzZvHMAybTnMA9SEZemtra1FR0dKlS4VC4Z49e6ysrGj5wA7CoOH+l21paWEY5vz5813WrK6unj59elpaWmJiYm1t7cWLF83Nzf39/dlZxdOnT7/xxht+fn43bty4d+/eH/7wh/fff3/dunX0Kl1lZWFh8eqrr9KFVp3Pw9m2bZtcLvf19aWL93s4subBgwfr16+PjIxEwicAGL4Qk0F/rVu3bvbs2ebm5jNmzGhra7Ozs4uLi7t79+7OnTsXLFggk8k8PT2/+uorhmFWrVrFd2d70dHRQeODXmueOHGCvt0RiUReXl4ODg7Xr19fvnw5W2H4DoKR4uLi7ty5s3v37sDAQCsrK3d397S0NCcnp+joaHqYNCFk9uzZcXFxNjY2dnZ2q1atWrp0aWJiovEJFDZt2qTX62/cuOHh4eHl5dVdoqm6urp58+bNnj176GcxBQDoAWIy6K9///d/NyjJysoyMTEJDAxkSxQKhaen5+XLl4dsHnOKfZvVa002gUVFRUV4eHhmZqbBadADNQj01WN7e3uXV9vb22mFwUcPGuIeOyMWi/39/VtaWr799ltCSGBgIHcqkxCiVCpbW1uvXbtm/F1EIpGHh8f+/fuDg4M3b9783XffGVTQ6XQajWbSpElHjx591qHonGpuxMrIyMjIyOC7FyPIM31RYeTAvkvoL+6sIiFEr9drtVpCSJfLsUtLS59TMli+jBkzJiUl5dKlS59//nlYWJi3tzcZ0EGQyWSEkO7eLTU0NLCzpYOJPqBEIjHYB0APkK6uriaEaLXaP/3pT5mZmRUVFQ0NDWwduoHgWQUFBWVmZmZnZ8+dO5ctbGtroycMHj58uA+xKc1SBoSQXbt2EULee+89vjsyUoSHh/PdBRiKEJPBABOLxdbW1s3NzS0tLX1Ol9+ZiYnJ06dPuSXcX/P8kkgkn376aXh4+Pr16+mLnAEcBHd3d0JIl++W9Hr9jz/+OHHixP603zdisVgul2u12qamJm5YRmctaeb0oKCgf/zjH4mJiUuWLLGzsxMIBLt3737vvfe4U8PGvzCgKTAM0r9FRUXp9frMzEx2kCdMmJCamjpz5kxj2sTiMxbdIYEBGTSIyaBLmLuEgRcaGtrW1paXl8ct3LFjh4uLS+d13Eai2b/YH6urq3/66ad+9XJAhYWFeXl55ebmnjlzhpb0cxDMzMzocUlubm4eHh4FBQWlpaUGddLT0+3t7XtY+f5cLVq0iBBy6tQptkSv1+fm5kqlUo1G097enpeXp1AooqOj7e3taezV0tJi0Ii5uTkbar/88st0/nf16tUREREGNXNycggh06dPZ0u2bNly7dq1EydO0HANAGC4Q0wGAy8+Pt7NzW3FihU5OTlarba+vj4pKWnr1q0JCQl9fmkUEBBQWVm5Z8+e5ubmsrKymJgYBweHge026dO+S0rw81mQ69evp++BBnAQdu3aZWJiMn/+/L/+9a/19fXt7e2VlZX79u179913d+7caWLCz9/i+Ph4V1fX2NjY7OzspqamkpKSpUuXVlVVJSYmOjo6mpqazp49u7q6+vPPP6+trW1paTl79mznNfivvPJKSUnJvXv38vPzb9++rVaraXlaWtrWrVvLy8v1en15efm6detSU1NVKlVkZCStkJKS8vHHH1+8eNHS0pK7TKesrGxQRwEAYAA978ObYDgivZ3F1jmhg0EFmrlq/PjxQqHQ3t4+ICDgzJkzxty6u8MBGxoaIiMjnZycpFKpr69vYWEhmwuju7MgDZw8ebLz99/gsEi1Wm1jY3PhwoXuGjFYPBceHs69yqbyovkdeh4Eg6Y6u3HjBlv58uXLERER48aNE4vFIpFo7NixYWFheXl5xjw4y5gzDenKfdayZcsYhnFzc+MW3rt3j1aura2NjY11dXUVCoVyuVyj0eTm5rJN1dTUREVFOTs7C4VCR0fHt956a/369bQFlUpF69y8eVOtVltYWDg7O9ND3BmG0Wq1ycnJGo2GpnaTyWQqlX1+eFgAACAASURBVCo+Pv7x48ds49y9BQby8/N7HYrBPO9yWMB5l4OM4LxL6IqAMWLbP4w0AoHg+PHjWFzyggkLCyM/rxwa4dLT02k8zXdHhgp8NwYZ/o2FLmHuEgAAns3du3eDg4MbGxtra2vZiWMvL68nT55wq3GvCgQCuit5SGEYJi8vb+XKle7u7mKx2MHBwdfXNzU1lRuvP3r06MCBA35+fqNGjZJKpRMnTly2bFlxcTG3nfXr12MbL/QfYjIAAHgGRUVF3t7eAQEBVlZWdnZ2DMMUFhbS8tjYWG5NejU/P9/W1pZhmEuXLvHU5W7dunXL19e3pKQkIyNDq9UWFBS4uLgsX758zZo1bJ01a9asWrUqJCTk+vXrdXV1hw4dKioqUqlUWVlZbJ233347Li7uww8/5OMh4MWBmAwGWw95FLds2TJ02gQYKDKZrIdjQ4d++1yNjY1BQUGvvfbau+++yy0Xi8W2trZJSUnHjh0bnJ4MFDMzs/T09KlTp0okkvHjx6ekpNja2u7Zs0ev17N1VqxYERMTo1AozM3N1Wp1Wlpae3v72rVr2Qpubm6ZmZnbt29PT0/n4yHgBYH8ZDDYnsciHiwMAhgcn332WXV19ebNmw3KJRLJ0aNHFyxYEBUVpVKpaF69oc/Dw6O1tZVbIhKJnJ2di4qKnjx5QtOssEe4spRKpVQqLSsrYxiGzbGnVCoXL178wQcfhIaGDmBqRhhR8J4MAACMwjBMcnLyjBkzRo8e3fmqRqPZtGlTU1NTWFiYwcKyYaShoaG0tNTLy6vLQzgonU7X0tIyefJkg6THixYtqqio4CbtA3gmiMkAALpFE5q4ubmJRCIbG5v58+ezh3hu27aNzo+z84anT5+mJXZ2drQkISFBIBDodLq8vDx6ib5BoeUCgWDs2LGFhYX+/v6Wlpbm5uZz5sxh8wz3p/3npLi4+MGDB0qlsrsKH330UUBAwJUrV1atWtVzUz0MbFZWFrv2oLy8PDw83Nra2tbWNjAw0CD/XE1NTXR0NM2ZYm9vHxoaWlRU1Oena2xszMvLCw4OVigUR44c6aEm3aC6ceNGg/Jp06YRQuh5rwD/X3t3H9bEle8B/AyQhJBAQORdVKBSblEjje6CJRcFl9iKUFkRq7Tb9dLL7XaLPFtai1q31VrWPq7VfdSKsl5tkVXWXumC0q5ldZ9HxBZ1wbcCLqhVIMhLCRAhFJn7x3k6zzRACG8ZXr6fv8yZkzNnTibk58yZ3xkKgXJwwJhGkDtnIkIOKo6Z+cnq6up8fHzc3Nzy8vJ0Ol1FRUVsbCzDMPy0djKZjKaj46hUKjql3UQdSqlUymSykJCQixcvtre3l5SUzJ07VywWnz9/fkTaX7x48ZQpU8xJ2GbmufHpp58SQj744AOj8pKSEoVCQf/d0NDg7e1NCKFPL7K8Of4ccwY2JiaGEBITE0MH5+zZs1KpdMGCBVyF2traGTNmuLm5nT59uq2t7caNG2FhYba2tibyC5qwbds2+pu4aNGia9eumaip1Wrd3NwSExN7b6Kr3KrV6gF3h7+x0CdcJwMA6FtaWtqdO3d2794dFRXl4ODg7++fnZ3t4eGRnJxMV/YcPr1ev3///pCQEJlMNn/+/KysrK6urvXr149I4z09PfQP/Yi0Rgipq6sjhJi4qUcImTp1ak5OjkgkSkpKouuD9Wb+wCYmJtLBWbJkybJly0pKShobG7lG7t27t2vXrueee04ulwcGBh4/fpxl2QEv0fVp8+bNBoPh22+/DQgICAoK4kI0I01NTUuXLl20aFHvRSkIIQ4ODgzD0FECGALEZAAAfaOrGvDXDJBIJBERER0dHSN1f0omk9EbXtScOXM8PT3LyspG5Hf9/Pnzzc3NISEhw2+KorPERCKR6WrBwcE7d+7U6/VxcXG9FzklgxlY/gqn9PJbbW0tfZmbm2tlZRUVFcVVcHd3DwwMvHLlyoMHDwZ7aIQQsVgcEBDw8ccfR0dHb9my5auvvjKqoNfrNRrNU089dezYMWtr6z4bsbGx6fOQAcyBmAwAoA8Gg0Gn09na2trb2/PL3dzcCCFarXZE9uLo6GhUQhdyffjw4Yi0P7JsbW0JIUYPKvYpOTk5Pj7+xo0bRikzyCAHln9NTiwWE0J6enq4Rnp6ehQKBT/3zdWrVwkht2/fHtoBUsuXLyeE5Ofn8wu7u7vj4uK8vLyOHj3aX0BGq0ml0uHsHSYzPK8LANAHiUSiUCh0Ol1bWxs/eqA319zd3elLKyurrq4u/htbWlqMmjJ6Oo+vqamJ5eVTID9GYzQyG377I8vDw4MQQmdNDSgzM7O0tPTw4cM0kuOYObCmSSQSR0fH9vb2jo6OEX+sgabAaG5u5hcmJSUZDIZTp05xu3viiSeysrKCg4O5Oq2trSzL0lECGAJcJwMA6NuKFSsIIfzUBgaDobCwUCqVajQaWuLh4VFTU8NV0Gq13333nVE7dnZ2XFz15JNPHjx4kNvU2dlJk+BT169fr62tVSqV3O/6MNsfWbNnzyaEmHlnUC6Xf/bZZzKZbP/+/UabzBnYAcXGxnZ3d3OPqVI7duyYPn16d3e3mY2kpqYmJCQYFRYUFJCf3jZ99913b968+fnnn9NwrT/0k6KjBDAEiMkAAPqWnp7u4+OTkpKSn5/f1tZWWVm5Zs2aurq6PXv20BtthJDIyMja2tq9e/e2t7dXVVWtX7+eu8TFefrppysrK+/fv19cXFxdXa1Wq7lNCoVi48aNxcXFer3+8uXLCQkJYrF4z549XIXhtB8eHu7s7Hzp0qWRGhClUunq6mq01KMJgYGBGRkZvcvNGdgBpaen+/n5rVu3rqCgQKfTNTc3Z2RkbN26defOndylrISEBIZh7ty5Y6Kd7OzsrVu33r1712Aw3L17d8OGDVlZWSqVKjExkVY4cuTIe++99/XXX9vb2/Pvkxol5iCE0EwckZGRZh4CgDEBn/mEMYvgOe2JCLkwOGbmwmBZtrGxMSUlxcfHRyQSKRQKjUZTWFjIr9DS0pKYmOjh4SGVSkNDQ0tKSlQqFf3rumHDBlqnvLxcrVbLZDJvb+99+/Zx71UqlV5eXrdu3dJoNPb29lKpNCws7MKFCyPVvlqtdnJyMic3hPnnxsaNG21sbGpqaujLhoYG/g+KSqXq/ZZXX33VKBcGa3Jgi4uL+W1u2rSJ/emjo8uWLaM1aZIzX19fkUjk4uISGRl59uxZ/l7Cw8Plcnl3d3d/h6PT6TIzMzUaDU1yJpfLVSpVenr6o0ePuDr8ZxGMGOUZoRPOurq6BhxG/I2FPjEsFqWBXhiGOXHixKpVq4TuCIykuLg48mO6y0kuJycnPj5e8L9+8+bNa2xsHNpDgiPL/HNDp9MFBgZGRUX1mQxiTGlpafH09Fy7du2hQ4cssLuysrKgoKDs7OzVq1cPWBl/Y6FPuHcJAADmUigUeXl5J0+e3Ldvn9B9MYVl2eTkZAcHh/4yjY2s6urq2NjYtLQ0cwIygP4gJgMAgEEICgq6fPlyQUFBa2ur0H3pV319fXV1dWFhoZkPcg5TRkbG9u3bt2/fboF9wQSGmAwAwNLoOpVlZWU1NTUMw2zevFnoHg3OzJkz8/PzHRwchO5Iv9zd3S9cuBAYGGiZ3e3YsQNXyGD4kJ8MAMDSUlNTU1NThe4FAIwtuE4GAAAAIDzEZAAAAADCQ0wGAAAAIDzEZAAAAADCwxx/6JtRKm2YAGhu0pycHKE7Ijx6emMoODg3AMYC5PGHPjAMI3QXAAAmMuTxh94QkwHAuIeVagBgAsB8MgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhISYDAAAAEB5iMgAAAADhMSzLCt0HAIDBSUpKqqio4F5evXrVx8fHycmJvrS2tj569Oi0adME6h0AwFDYCN0BAIBBc3NzO3jwIL/k2rVr3L99fX0RkAHAuIN7lwAw/qxZs6a/TWKx+OWXX7ZgXwAARgbuXQLAuDR79uxbt271+ResoqLC39/f8l0CABgOXCcDgHHppZdesra2NipkGEapVCIgA4DxCDEZAIxLL7zwwuPHj40Kra2tf/WrXwnSHwCAYcK9SwAYrxYuXPj111/39PRwJQzD3L9/38vLS8BeAQAMDa6TAcB49eKLLzIMw720srIKDQ1FQAYA4xRiMgAYr+Li4vgvGYZ56aWXhOoMAMAwISYDgPFq6tSpERER3Ex/hmFWrFghbJcAAIYMMRkAjGMJCQl0Uqy1tbVGo3F2dha6RwAAQ4SYDADGsdjYWLFYTAhhWTYhIUHo7gAADB1iMgAYx2QyWVRUFCFELBYvX75c6O4AAAwdYjIAGN/Wrl1LCFmxYoVMJhO6LwAAQ4f8ZBaSk5MTHx8vdC8AAAAGZ+XKlX/961+F7sWkYCN0ByaXEydOCN0FgHGguLh49+7d5n9fsrKyVq9ebWMzMf+gxcfHp6SkhISECN2RMWGw5wYM00cffSR0FyYRXCezEHqdDKMNYI7Bfl86OzttbW1HtUsCYhjmxIkTq1atErojYwL+lloYzQKI62SWgflkADDuTeCADAAmD8RkAAAAAMJDTAYAAAAgPMRkAAAAAMJDTAYAABPWvXv3oqOjW1tbGxsbmR8FBQV1dnbyq/G3Mgwzf/58oTrcH5Zli4qKXnvtNX9/f4lE4urqGhoampWVxX/c4fvvvz9w4EB4ePiUKVOkUumsWbPWrl1bVlbGb+ftt9/GU6tjFmIyAIAJqL29fdasWXSRg0mrtLR0/vz5kZGRDg4OU6dOZVm2pKSElqekpPBr0q3FxcXOzs4sy16+fFmgLveroqIiNDS0srLy5MmTOp3u0qVL06dPf/HFF998802uzptvvvn666/HxMTcunWrqanp8OHDpaWlKpUqNzeXq/PKK6+kpaW98847QhwEDAAxGQDABMSybE9PT09Pj1AdkMvloaGhQu2dENLa2rp8+fJf/vKXv/3tb/nlEonE2dk5IyPjL3/5i1B9GxobG5ucnJy5c+fa2tr6+voeOXLE2dl57969BoOBq7Nu3br169e7u7vb2dmp1ers7OzHjx+/9dZbXAU/P79Tp05t3749JydHiIMAUxCTAQBMQPb29lVVVWfOnBG6I4L58MMPtVrtli1bjMptbW2PHTtmZWWVlJRUWVkpSN+GICAg4IcffnBycuJKxGKxt7e3wWDg7sNmZmZmZGTw36VUKqVSaVVVFf8Wp1KpXLly5RtvvNHd3W2ZzoOZEJMBAMBEw7JsZmbmz3/+c09Pz95bNRrN5s2b29ra4uLijCaWjSMtLS23b98OCgpSKBT91dHr9R0dHbNnz2YYhl++YsWKBw8enD59evS7CYOAmAwAYKLJzc3lpqvTmINfcvfu3fj4eEdHR2dn56ioqKqqKvqunTt30grTpk0rKSmJiIiwt7e3s7NbvHhxUVERrfP+++/TOtx9yS+++IKWTJ06ld+OXq8vKiqimyy/7FVZWVl9fb1Sqeyvwu9///vIyMhr1669/vrrpptqamr63e9+5+fnJxaLnZycnn322XPnztFN5owq1dDQkJycPHPmTLFY7OLiEhsbW1paOuSja21tLSoqio6Odnd3/+STT0zUpPn3N23aZFQ+b948QsiXX3455D7AqGDBIuhzLkL3AmB8wPeFjxBy4sSJIbwxJiaGENLR0WFUEhMTc/Hixfb29rNnz0ql0gULFvDfpVQqZTJZSEgIrVNSUjJ37lyxWHz+/Hmujkwme+aZZ/jvUqlUdHa8iTrU4sWLp0yZUlxcPIQjYs0+Nz799FNCyAcffGBUXlJSolAo6L8bGhq8vb0JIfTpRZY3x59TV1fn4+Pj5uaWl5en0+kqKipiY2MZhjl06BBXZ8BRra2tnTFjhpub2+nTp9va2m7cuBEWFmZra3vx4sUhjMC2bdvoz/eiRYuuXbtmoqZWq3Vzc0tMTOy9SafTEULUavWAu1u5cuXKlSuH0E8YAlwnAwCYXBITE0NCQmQy2ZIlS5YtW1ZSUtLY2MivoNfr9+/fT+vMnz8/Kyurq6tr/fr1I7L3np4e+vMzIq31p66ujhBi4qYeIWTq1Kk5OTkikSgpKam8vLzPOmlpaXfu3Nm9e3dUVJSDg4O/v392draHh0dycnJ9fT2/polRTUtLu3fv3q5du5577jm5XB4YGHj8+HGWZQe8RNenzZs3GwyGb7/9NiAgICgoiAvRjDQ1NS1dunTRokUHDhzovdXBwYFhGDpKMHYgJgMAmFwWLFjA/ZteKKqtreVXkMlk9N4WNWfOHE9Pz7KyshH5CT9//nxzc3NISMjwmzKB3rEViUSmqwUHB+/cuVOv18fFxXV0dPSucOrUKULIsmXLuBKJRBIREdHR0WF048/EqObm5lpZWfHzkri7uwcGBl65cuXBgweDPTRCiFgsDggI+Pjjj6Ojo7ds2fLVV18ZVdDr9RqN5qmnnjp27Ji1tXWfjdjY2PR5yCAgxGQAAJML/+qRWCwmhBilzHB0dDR6i6urKyHk4cOHo9+7kUGXpf/hhx8GrJmcnBwfH3/jxg2jlBmEEIPBoNPpbG1t7e3t+eVubm6EEK1Wyy/sb1RpIz09PQqFgp+W9urVq4SQ27dvD+0AqeXLlxNC8vPz+YXd3d1xcXFeXl5Hjx7tLyCj1aRS6XD2DiPO0vMuAQBgjGtqamJZlv+kHo3GaGRGCLGysurq6uK/paWlxagRowf9LMzDw4MQQmdNDSgzM7O0tPTw4cM0kuNIJBKFQqHT6dra2vhhGb1r6e7ubk7jEonE0dGxvb29o6NjxJ91kEgkhJDm5mZ+YVJSksFgOHXqFLe7J554IisrKzg4mKvT2trKsiwdJRg7cJ0MAAB+orOzk+a7p65fv15bW6tUKrmfcA8Pj5qaGq6CVqv97rvvjBqxs7Pj4rYnn3zy4MGDo9zrn5g9ezYhxMw7g3K5/LPPPpPJZPv37zfatGLFCkIIP2eEwWAoLCyUSqUajcbMzsTGxnZ3d3PPrlI7duyYPn26+RnCUlNTExISjAoLCgrIT2+bvvvuuzdv3vz8889puNYf+vHRUYKxAzEZAAD8hEKh2LhxY3FxsV6vv3z5ckJCglgs3rNnD1chMjKytrZ279697e3tVVVV69ev5y6hcZ5++unKysr79+8XFxdXV1er1WpaHh4e7uzsfOnSpVE9BKVS6erqarTUowmBgYFG2Vap9PR0Hx+flJSU/Pz8tra2ysrKNWvW1NXV7dmzh97BNEd6erqfn9+6desKCgp0Ol1zc3NGRsbWrVt37tzJXcpKSEhgGObOnTsm2snOzt66devdu3cNBsPdu3c3bNiQlZWlUqkSExNphSNHjrz33ntff/21vb09/z6pUWIOQgjNxBEZGWnmIYCFCPjM56SCZ/sBzIfvCx8ZfC4MOjOds3bt2uLiYn7Jpk2b2J8++bhs2TL6XqVS6eXldevWLY1GY29vL5VKw8LCLly4wG+/paUlMTHRw8NDKpWGhoaWlJSoVCrazoYNG2id8vJytVotk8m8vb337dvHvVetVjs5OQ0tDQQ7mHNj48aNNjY2NTU19GVDQwP/eFUqVe+3vPrqq0a5MFiWbWxsTElJ8fHxEYlECoVCo9EUFhbSTeaPKk1y5uvrKxKJXFxcIiMjz549y99LeHi4XC7v7u7u73B0Ol1mZqZGo6FJzuRyuUqlSk9Pf/ToEVeH/yyCEaPkI3TCWVdX14DDiFwYlsSwo/xAMlA5OTnx8fEYbQBz4PvCxzDMiRMnVq1aZZndzZs3r7GxcWjPA1qA+eeGTqcLDAyMiorqMxnEmNLS0uLp6bl27dpDhw5ZYHdlZWVBQUHZ2dmrV68esHJcXBz5MfcsjDbcu4SJqXcec8s4fvw43anRZGFznDlzxt/ff8izgOVyOf9uhZWVlZOTk1Kp/M1vfnPlypWhtTku3Lt379e//vX06dPFYjF3+O+//74Fds2ybFFR0Wuvvebv7y+RSFxdXUNDQ2kCUq7O999/f+DAgfDw8ClTpkil0lmzZq1du9b8e2owZAqFIi8v7+TJk/v27RO6L6awLJucnOzg4NBfprGRVV1dHRsbm5aWZk5ABhaGmAwsob29fdasWfz0PKPt+eefZ3/Mr21Jq1evZlk2IiJiUO+qqqqKjo5OS0szykI5KO3t7f/6178IITExMSzL/vDDD+Xl5Vu3bi0vL58/f/6vf/3rR48eDbnxMauhoSE4OPjq1as5OTktLS1sr9tJo6qioiI0NLSysvLkyZM6ne7SpUvTp09/8cUX33zzTa7Om2+++frrr8fExNy6daupqenw4cOlpaUqlSo3N9di/Zy0goKCLl++XFBQ0NraKnRf+lVfX19dXV1YWGjmg5zDlJGRsX379u3bt1tgXzBYiMnAEliW7enpMcqBBJx33nln4cKFV65cMUqDNBzW1tZubm4xMTH/+Mc/3nrrrSNHjrzwwgsT725gZmamVqv96KOPgoOD7ezsRm9HcrmcW+GRz8bGJicnZ+7cuba2tr6+vkeOHHF2dt67d6/BYODqrFu3bv369e7u7nZ2dmq1Ojs7+/Hjx2+99dbo9XZo6DqVZWVlNTU1DMNs3rxZ6B6NgJkzZ+bn5zs4OAjdkX65u7tfuHAhMDDQMrvbsWMHrpCNWchPBpZgb2/f+8Ef4Pz5z38e1eSNf/jDH/75z3/+7W9/O378+AsvvDB6O7K869evE0LmzJkjyN4DAgKMspKKxWJvb+/S0tLOzk6ajCAzM9PoXUqlUiqVVlVVsT/NASa41NTU1NRUoXsBMHnhOhmA8EY7mzbDMDRHee/0S+MdvSE7gtcXh6mlpeX27dtBQUEmVlrU6/UdHR2zZ88eUwEZAAgOMdkYwp+WXlFRsWrVKmdnZ/qSrmXb0NCQnJxMH4R2cXGJjY2lOWY45eXlzz//vEKhsLOz+9nPfpafn79kyRLaQmJi4jDbNxgMW7ZsCQgIsLOzmzJlyvLly//2t789fvx4wK39TbenD4f7+fmJxWInJ6dnn3323LlzvYfi7t278fHxjo6Ozs7OUVFRQ7jeptVqTbRg+qi7u7tPnDjxi1/8wt3dXSqVzpkzZ8+ePUY3Yblhl8lkarX6woULg+2hBdD7bpcuXeKu65g4cDPH3/QpYXoXw0c7+fnnnxNCpFIpwzB93lukTJxsZKBPmd7R0+v1RUVFdEz6fA6jtbW1qKgoOjra3d39k08+MdFz+gjbpk2bhnbgADBhCZGAYzIyP6cOnZYeFhZ27tw5vV5/6dIla2vrhoaG2traGTNmuLm5nT59uq2t7caNG2FhYba2tlyan9u3bzs6Onp5ef3973+nFZYsWeLi4iKRSEak/cTERIVC8fe///3Ro0darZbe4zh37pw5W7n9dnR00Jd1dXU+Pj5ubm55eXk6na6ioiI2NpZhmEOHDhm9JSYm5uLFi+3t7WfPnpVKpQsWLDB/2I1aKCwsdHBw4Lcw4FHn5eURQj744IPm5uaGhoY//elPVlZWqampXAtGw37t2rXIyMiZM2caDbuZvLy8rK2t+9y0ePHiKVOmGCUZMsKf42+EW2y4trbWnANnzRh/0x+6Obvoz2C/L9x5xf44x3/btm1cyYAn24CfMsuyMpnsmWee6a8b3BNzixYtunbtmokOa7VaNze3xMREc46OIoPPTzaBIXedhSE/mSXhzLaQwf7GnDlzxqj8V7/6FSHk2LFjXEldXZ1EIuEyH9IsMidPnuQqPHz40M7Ors+YbAjt+/j4LFy4kP8Wf39/7gfY9Fa212/nyy+/TAj5y1/+wlXo7Oz09PSUSqVarZb/lry8PK7OypUrCSENDQ2seXq3sGbNGn4LAx51Xl7eokWL+G0mJCSIRCKdTkdf9h72mpoaiUQy4jFZWFjYgJk2TcRk3EOXNCYb8MBZM8bf9Iduzi76M7Ix2YAn24CfMjtQTMayrMFg+Pbbb//nf/7H2tp669atfdZpbGycN29efHy8idSgvSEm40NMZmGIySwJc/zHqJ/97GdGJbm5uVZWVvx0Eu7u7oGBgVeuXHnw4MG0adO++OILQgh/CTYXF5eAgICbN2+OSPtLly79+OOP//u//3vdunULFiywtrauqKjgKpve2hvNM87POi2RSCIiIj799NMvv/zypZde4sr5S7l5e3sTQmpra6dOnWqicSP8Fry8vPgtDHjUUVFRRik8lEplVlbWzZs3Q0JCCCG9h93T09Pf37+ystL8Hprj/Pnzw3l7XV0dIUQkEpl54Fy5ifE3/aGbv4vRNuDJNuCnbA6xWBwQEPDxxx/X19dv2bIlJCRkyZIl/Ap6vV6j0Tz11FOffPKJtbX1oA7Bkgk+xjg6FDk5OUJ3ZLKw8Ld1kkNMNkbJZDL+S4PBoNPpCCF9Thy+ffu2i4tLW1ubra2tXC7nb3JychqR9qdNm7Zv376QkJCjR4/S5FtqtTopKYku0EsIMb3VCN2dra2t0dRsun6cVqvlF/K7JBaLCSGDzanBb8HKyoprwZyj1ul0f/zjH0+dOvXgwYOWlhZuK73sZDAY+hx2V1fXEY/JhonOcgsJCRGJROYcOPfSxPib+NAHtYtRZc7JZvpTHqzly5efOnWKzubkCru7u+lqNkePHh1sQEYI2b179+7du4fQmYkqPj5e6C5MIvQCOVgA5viPDxKJFrHipAAADTZJREFUxNHR0cbG5ocffuh9tXPx4sUSicTe3r6zs7O9vZ3/xocPH45I+4QQhmFefPHFr776qqWlJTc3l2XZ2NjYXbt20RZMb+29O4VC0dnZ2dbWxi+nGVMtkziRmHfUy5cv37Zt2yuvvFJZWdnT08Oy7EcffUQIYVmWttDnsDc3N1vmEMzU09NDU5m/9tprxLwDN4eJD32kdjF85pxspj9l7mDN3yPpdQ4kJSUZDIacnBzu+YAnnnjC/HW4ce+Sg3uXFoaAzJIQk40bsbGx3d3dRUVF/MIdO3ZMnz69u7ubEPLss8+SH2+lUVqt1vyrNQO27+joWF5eTggRiUS/+MUv6FNvp0+fpjVNb+2NXk3hVzAYDIWFhVKplH8fcLSZPurHjx8XFRW5u7snJye7uLjQX2VusjzVe9gbGxtN37e1vLS0tG+++WbFihV09hsx4+M2h+kPfUR2MSJMn2zmfMqEEDs7u66uLvrvJ5988uDBg4SQ1NTUhIQEo5oFBQXkp/d833333Zs3b37++ec0XAMA6JvQIfhkMZw5y1R9fb2fn5+vr++ZM2daWlqampoOHDhgZ2fH/Qf63//+95QpU7gHAK9fv7506dIZM2b0Ocd/CO0rFIqwsLCysrLOzs76+vp3332XEPL++++bs7X3fvmPwrW2tnKPwh08eNBEVzds2EAI+de//mXOSJrTwoBHHR4eTgj58MMPGxoaHj169I9//GP69OmEkLNnz/Y57Ddv3tRoNK6uroI/d/n48eP6+vrc3Fx6COvWrXv06BFXc8ADN2f0TH/o5uyiP6P33GWfJ9uAnzLLskuXLlUoFN99993FixdtbGxu3brFsuwbb7zBMMx77713586dzs7OO3fu0Oz8KpWKG+3//d//7e/Pr+lPk0NwnYwH18ksDHP8LQlntoWY83ek9zReowo0x5Kvr69IJHJxcYmMjOT/ZrAsW1FR8fzzzzs4ONjZ2S1cuPCf//znokWL7OzsRqT90tLSpKSk//iP/6DJqIKDgw8dOkRv9JjeSmdYc9auXUvf0tjYmJKS4uPjIxKJFAqFRqMpLCzss6ubNm1if7oo0LJlywY1mCZaMH3UDQ0NSUlJ3t7eIpHIzc3t5Zdffvvtt2kL3POD3LDTVBH5+fncepf/9V//ZbqfFM3FYISfFoRlWbVabfq5S6M5ggzDKBSKOXPmvPrqq1euXOld38SBmzl6pk+JAcfWBHO+L32eV35+fvzC+/fv08omTjbWvE+5vLxcrVbLZDJvb+99+/bRQp1Ol5mZqdFoaA42uVyuUqnS09P54S//2QIjiMmGADGZhSEmsySGnXDr341NOTk58fHxlh/tgICAjo6Oe/fuWXi/AMMh1PdlbGIY5sSJE6tWrRK6I2MCzg0LoxMeaKJjGG2YTzZxaLXaKVOm8Fffu3v3blVVFb0vAwAAAGMZYrIJ5fvvv09KSrp///6jR4+++eab+Ph4BweHd955R+h+AQCMFffu3YuOjm5tbW1sbOTWEAsKCuKv/EYI4W9lGGb+/PlCdXhAZ86c8ff373PJr7fffpve7YVxATHZxOHu7k6zEvznf/6nk5NTdHT0rFmzvvnmG19fX6G7NiqY/tHJ5mPEeOknwGRQWlo6f/78yMhIBweHqVOnsixbUlJCy1NSUvg16dbi4mJnZ2eWZS9fvixQl02pqqqKjo5OS0ujuV16e+WVV9LS0vA/8/ECOWMnlIiICG52+YQ3XiaUjJd+AhBC5HL5vHnzaIbh8di+aa2trcuXL//lL3/529/+ll8ukUjkcnlGRkZYWNgLL7wgSN+G5p133lm4cOH//d//zZw5U6/X967g5+d36tSpoKCgOXPmYEri2IfrZAAAMCl8+OGHWq12y5YtRuW2trbHjh2zsrJKSkoaaytwmPbnP//57bff7vOuJUepVK5cufKNN96wcF5AGALEZAAAMPGxLJuZmfnzn//c09Oz91aNRrN58+a2tra4uDijiWVjmVQqNafaihUrHjx4YCKJN4wRiMkAACYCmg3Oz89PLBY7OTk9++yz586do5vef/99OoUxNDSUlnzxxRe0hC4nTwjZuXMnwzB6vb6oqIhuoldfaDnDMNOmTSspKYmIiLC3t7ezs1u8eDG3SMNw2reYsrKy+vp6pVLZX4Xf//73kZGR165de/311003ZWKo6WoW1N27d+Pj4x0dHZ2dnaOioqqqqviNNDQ0JCcn08x2Li4usbGxpaWlwz/MPs2bN48Q8uWXX45S+zBihEqMNtkgzyGA+fB94SNm5Izlr1Wg0+m4tQr4mYdlMtkzzzzDf5dKpaIT2E3UoZRKpUwmCwkJuXjxYnt7e0lJydy5c8Vi8fnz50ekfXOWqaCGfG58+umnhJAPPvjAqLykpEShUNB/NzQ0eHt7E0KysrJoCTfHn2POUNPlJWJiYuhwnT17liaU5irU1tbOmDHDzc3t9OnTbW1tN27cCAsLs7W1NZEU2jQTS4CwLKvT6QgharV6CC0jZ6wl4ToZAMC4l5aWdufOnd27d0dFRTk4OPj7+2dnZ3t4eCQnJ/f3RN5g6fX6/fv3h4SEyGSy+fPnZ2VldXV1rV+/fkQa55Z/GJHW+lRXV0cIUSgUJupMnTo1JydHJBIlJSXRtVx7M3+oExMT6XAtWbJk2bJlJSUljY2NXCP37t3btWvXc889J5fLAwMDjx8/zrLsgJfohsbBwYFhGDoCMJYhJgMAGPfoSlP8dZwkEklERERHR8dI3bGSyWT0Fhg1Z84cT0/PsrKyEfmlP3/+fHNzc0hIyPCb6g+dJSYSiUxXCw4O3rlzp16vj4uL670UPRnMUPPXoaeX32pra+nL3NxcKyurqKgoroK7u3tgYOCVK1cePHgw2EMzh42NTZ+HA2MKYjIAgPHNYDDodDpbW1t7e3t+uZubGyFEq9WOyF4cHR2NSlxdXQkhDx8+HJH2R5utrS0hhL/SSX+Sk5Pj4+Nv3LhhlDKDDHKo+dfkxGIxIaSnp4drpKenR6FQ8BMWXr16lRBy+/btoR2gad3d3WY+EAACQn4yAIDxTSKRKBQKnU7X1tbGjxXorTR3d3f60srKqquri//GlpYWo6YYhulvL01NTSzL8ivQaIxGZsNvf7R5eHgQQujMqgFlZmaWlpYePnyYRnIcM4faNIlE4ujo2N7e3tHRYZkHHVpbW1mWpSMAYxmukwEAjHsrVqwghPCTHRgMhsLCQqlUqtFoaImHh0dNTQ1XQavVfvfdd0bt2NnZcXHVk08+efDgQW5TZ2cnTXlPXb9+vba2VqlUcr/0w2x/tM2ePZsQYuadQblc/tlnn8lksv379xttMmeoBxQbG9vd3c09uErt2LFj+vTpo5FFjH4udARgLENMBgAw7qWnp/v4+KSkpOTn57e1tVVWVq5Zs6aurm7Pnj30thohJDIysra2du/eve3t7VVVVevXr+cucXGefvrpysrK+/fvFxcXV1dXq9VqbpNCodi4cWNxcbFer798+XJCQoJYLN6zZw9XYTjth4eHOzs7X7p0aeSH5kdKpdLV1bWsrMzM+oGBgRkZGb3LzRnqAaWnp/v5+a1bt66goECn0zU3N2dkZGzdunXnzp3clbOEhASGYe7cuWNmmybQLBuRkZHDbwpGl4DPfE4qeLYfwHz4vvARM3JhsCzb2NiYkpLi4+MjEokUCoVGoyksLORXaGlpSUxM9PDwkEqloaGhJSUlKpWK/hBs2LCB1ikvL1er1TKZzNvbe9++fdx7lUqll5fXrVu3NBqNvb29VCoNCwu7cOHCSLWvVqudnJzMyQQxnHNj48aNNjY2NTU19GVDQwP/11ClUvV+y6uvvmqUC4M1OdTFxcX8Njdt2sT+9GHSZcuW0Zo0yZmvr69IJHJxcYmMjDx79ix/L+Hh4XK5vLu728QR5eXl9f5Z52floOLi4ry8vLq6uswdKR7kwrAkhsVifBaRk5MTHx+P0QYwB74vfAzDnDhxQtjFCufNm9fY2DhKjwQOynDODZ1OFxgYGBUVdeDAgRHv2MhqaWnx9PRcu3btoUOHhtlUWVlZUFBQdnb26tWrh/D2uLg4Qshf//rXYXYDzIF7lwAAMCkoFIq8vLyTJ0/u27dP6L6YwrJscnKyg4PDtm3bhtlUdXV1bGxsWlra0AIysDDEZAAAMFkEBQVdvny5oKCgtbVV6L70q76+vrq6urCw0MwHOU3IyMjYvn379u3bR6RjMNoQkwEAQL/oOpVlZWU1NTUMw2zevFnoHg3XzJkz8/PzHRwchO5Iv9zd3S9cuBAYGDj8pnbs2IErZOMI8pMBAEC/UlNTU1NThe4FwKSA62QAAAAAwkNMBgAAACA8xGQAAAAAwkNMBgAAACA8zPG3KJp8DwBMo7lJ8X3hfPTRR0jaSeHcsLBLly4FBwcL3YvJAnn8LaS4uHjXrl1C9wIAAGBwQkJCfve73wndi0kBMRkAAACA8DCfDAAAAEB4iMkAAAAAhIeYDAAAAEB4iMkAAAAAhPf/h4NO2fXrtXgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 456 ms (started: 2021-07-25 13:52:50 +08:00)\n"
     ]
    }
   ],
   "source": [
    "plot_model(model, show_dtype=True, show_shapes=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "spread-monthly",
   "metadata": {},
   "source": [
    "### 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "attempted-memphis",
   "metadata": {},
   "source": [
    "以下是您应该从这个标量回归示例中获得的信息：\n",
    "* 回归使用与我们用于分类的损失函数不同的损失函数完成。均方误差 (MSE) 是通常用于回归的损失函数。\n",
    "* 同样，用于回归的评估指标与用于分类的评估指标不同； 自然，准确率的概念不适用于回归。 一个常见的回归指标是平均绝对误差 (MAE)。\n",
    "* 当输入数据中的特征具有不同范围的值时，每个特征都应作为预处理步骤独立缩放。\n",
    "* 当可用数据很少时，使用 K 折验证是可靠评估模型的好方法。\n",
    "* 当可用的训练数据很少时，最好使用中间层很少（通常只有一两个）的小模型，以避免严重的过拟合。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wooden-strength",
   "metadata": {},
   "source": [
    "## 本章总结"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "essential-emission",
   "metadata": {},
   "source": [
    "* 矢量数据上三种最常见的机器学习任务是二元分类、多类分类和标量回归。\n",
    "    - 本章前面的“总结”部分总结了您在每项任务中学到的要点。\n",
    "    - 回归使用与分类不同的损失函数和不同的评估指标。\n",
    "* 在将原始数据输入神经网络之前，您通常需要对其进行预处理。\n",
    "* 当您的数据具有不同范围的特征时，作为预处理的一部分，独立缩放每个特征。\n",
    "* 随着训练的进行，神经网络最终开始过度拟合并在从未见过的数据上获得更糟糕的结果。\n",
    "* 如果你的训练数据不多，请使用只有一两个中间层的小模型，以避免严重的过拟合。\n",
    "* 如果你的数据被分成很多类别，中间层太小可能会造成信息瓶颈。\n",
    "* 当您处理少量数据时，K 折验证可以帮助可靠地评估您的模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "horizontal-speaking",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:tensorflow]",
   "language": "python",
   "name": "conda-env-tensorflow-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "toc-showcode": false,
  "toc-showmarkdowntxt": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
